Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rbd: migration secret support #2562

Merged
merged 5 commits into from
Oct 27, 2021
Merged

rbd: migration secret support #2562

merged 5 commits into from
Oct 27, 2021

Conversation

humblec
Copy link
Collaborator

@humblec humblec commented Oct 11, 2021

rbd: parse migration secret and set it for CSI driver operations

The intree secret has a data field called "key" which is the base64
admin secret key. The ceph CSI driver currently expect the secret to
contain data field "UserKey" for the equivalant. The CSI driver also
expect the "UserID" field which is not available in the in-tree secret
by deafult. This missing userID will be filled (if the username differ
than 'admin') in the migration secret as 'adminId' field in the
migration request, this commit adds the logic to parse this migration
secret as below:

"key" field value will be picked up from the migraion secret to "UserKey"
field.
"adminId" field value will be picked up from the migration secret to "UserID"
field

if `adminId` field is nil or not set, `UserID` field will be filled with
default value ie `admin`.The above logic get activated only when the secret
is a migration secret, otherwise skipped to the normal workflow as we have
today.

Note:
E2E tests are added to make sure migration kind of secret is passed in tests.

Additional note:
The initial plan was to ask the admin to adjust the secret as a pre-upgrade step
for migration which avoid this change in the CSI driver, however taking out admin burden
is the main ask from kubernetes community so this change is introduced. None of the
existing workflow is broken for CSI driver with this change.

Updates #2509

Signed-off-by: Humble Chirammal [email protected]

@humblec humblec marked this pull request as draft October 11, 2021 17:08
@mergify mergify bot added the component/rbd Issues related to RBD label Oct 11, 2021
@humblec humblec force-pushed the mig-secret branch 3 times, most recently from df32435 to a1231c8 Compare October 12, 2021 05:18
@Madhu-1
Copy link
Collaborator

Madhu-1 commented Oct 12, 2021

@humblec this was not part of the original plan as per #2509 (comment) why we are adding this one? Do we have a clear design doc/plan about the changes required in cephcsi? IMHO we are adding more complexity to cephcsi to support migration.

@humblec
Copy link
Collaborator Author

humblec commented Oct 12, 2021

@humblec this was not part of the original plan as per #2509 (comment) why we are adding this one? Do we have a clear design doc/plan about the changes required in cephcsi? IMHO we are adding more complexity to cephcsi to support migration.

@Madhu-1 I will add details about this change in the PR description and commit ..etc, thats why I kept this in draft atm . I have already added this change in the tracker and can add some more details there as well which explain how this helps. This looks to be pretty much the last change (unless some minor change comes up while doing complete e2e testing which cover kube.etc). To give some context the initial plan was to ask this change as part of the pre upgrade step, but found a way to help admin to stay away from this change mostly and we can take care of it from csi driver code itself without breaking any existing functionality. Reg# documentation, it is there in the todo as well. Thanks 👍

@humblec humblec force-pushed the mig-secret branch 7 times, most recently from ce589be to 903bdf9 Compare October 13, 2021 11:03
@humblec humblec added this to the release-3.5.0 milestone Oct 13, 2021
@humblec humblec requested a review from Madhu-1 October 13, 2021 11:10
@humblec humblec marked this pull request as ready for review October 13, 2021 11:13
@humblec
Copy link
Collaborator Author

humblec commented Oct 13, 2021

/retest ci/centos/mini-e2e-helm/k8s-1.20

@humblec
Copy link
Collaborator Author

humblec commented Oct 13, 2021

/retest ci/centos/mini-e2e-helm/k8s-1.22

@humblec humblec force-pushed the mig-secret branch 2 times, most recently from 50ce1af to 903bdf9 Compare October 15, 2021 06:36
@ceph ceph deleted a comment from mergify bot Oct 15, 2021
Copy link
Collaborator

@Madhu-1 Madhu-1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@humblec as its a migration work, why not ask the admin to recreate the secret and use it or take another route as we did for clusterID generation? IMO these changes make the cephcsi logic ugly to handle the migration.

e2e/ceph_user.go Outdated Show resolved Hide resolved
e2e/migration.go Outdated Show resolved Hide resolved
e2e/migration.go Outdated Show resolved Hide resolved
e2e/migration.go Outdated Show resolved Hide resolved
e2e/migration.go Outdated Show resolved Hide resolved
e2e/migration.go Outdated Show resolved Hide resolved
internal/rbd/controllerserver.go Outdated Show resolved Hide resolved
internal/rbd/controllerserver.go Outdated Show resolved Hide resolved
internal/rbd/migration.go Outdated Show resolved Hide resolved
@humblec
Copy link
Collaborator Author

humblec commented Oct 19, 2021

@humblec as its a migration work, why not ask the admin to recreate the secret and use it or take another route as we did for clusterID generation? IMO these changes make the cephcsi logic ugly to handle the migration.

@Madhu-1 as mentioned in the other PR comment #2562 (comment), this was the initial thought ie let admin patch the secret. Then after some discussions with community folks , it was recommended that, take care of it maximum from the CSI side . So this PR, the advantage with this PR is that, admin Dont want to patch the secret at all, if the admin user is not different from admin which is mostly the case.

@Madhu-1 Madhu-1 requested a review from nixpanic October 19, 2021 05:41
@humblec humblec force-pushed the mig-secret branch 2 times, most recently from 539ecc3 to 6f723cf Compare October 19, 2021 06:47
yati1998
yati1998 previously approved these changes Oct 27, 2021
e2e/migration.go Show resolved Hide resolved
@Madhu-1
Copy link
Collaborator

Madhu-1 commented Oct 27, 2021

@humblec please don't dismiss the requested review as am reviewing the PR. this can make the PR to be merged

@Madhu-1 Madhu-1 added the DNM DO NOT MERGE label Oct 27, 2021
e2e/migration.go Show resolved Hide resolved
e2e/rbd.go Show resolved Hide resolved
internal/rbd/migration.go Outdated Show resolved Hide resolved
internal/rbd/controllerserver.go Outdated Show resolved Hide resolved
internal/util/credentials.go Outdated Show resolved Hide resolved
internal/util/credentials.go Outdated Show resolved Hide resolved
@mergify mergify bot dismissed stale reviews from nixpanic and yati1998 October 27, 2021 10:19

Pull request has been modified.

@humblec humblec requested a review from Madhu-1 October 27, 2021 10:21
@humblec
Copy link
Collaborator Author

humblec commented Oct 27, 2021

/retest ci/centos/mini-e2e/k8s-1.21

1 similar comment
@humblec
Copy link
Collaborator Author

humblec commented Oct 27, 2021

/retest ci/centos/mini-e2e/k8s-1.21

@humblec
Copy link
Collaborator Author

humblec commented Oct 27, 2021

/retest ci/centos/mini-e2e-helm/k8s-1.22

@humblec
Copy link
Collaborator Author

humblec commented Oct 27, 2021

@Mergifyio rebase

This commit adds a couple of helper functions to parse the migration
request secret and set it for further csi driver operations.

More details:

The intree secret has a data field called "key" which is the base64
admin secret key. The ceph CSI driver currently expect the secret to
contain data field "UserKey" for the equivalant. The CSI driver also
expect the "UserID" field which is not available in the in-tree secret
by deafult. This missing userID will be filled (if the username differ
than 'admin') in the migration secret as 'adminId' field in the
migration request, this commit adds the logic to parse this migration
secret as below:

"key" field value will be picked up from the migraion secret to "UserKey"
field.
"adminId" field value will be picked up from the migration secret to "UserID"
field

if `adminId` field is nil or not set, `UserID` field will be filled with
default value ie `admin`.The above logic get activated only when the secret
is a migration secret, otherwise skipped to the normal workflow as we have
today.

Signed-off-by: Humble Chirammal <[email protected]>
…MigSecret

This commit adds unit tests for newly introduced migration specific
functions.

Signed-off-by: Humble Chirammal <[email protected]>
parseAndDeleteMigratedVolume() prviously clubbed the logic of
parsing of migration volume handle and then continued with the
deletion of the volume. however this commit split this
logic into two, ie parsing has been done in parseMigrationVolID()
and DeleteMigratedVolume() deletes the backend volume.

Signed-off-by: Humble Chirammal <[email protected]>
this commit make use of the migration request secret parsing and set
the required fields for further nodestage operations

Signed-off-by: Humble Chirammal <[email protected]>
this commit create and make use of migration secret in the requests and
validate various csi operations

Signed-off-by: Humble Chirammal <[email protected]>
@mergify
Copy link
Contributor

mergify bot commented Oct 27, 2021

rebase

✅ Branch has been successfully rebased

@humblec
Copy link
Collaborator Author

humblec commented Oct 27, 2021

/retest all

@humblec humblec removed the DNM DO NOT MERGE label Oct 27, 2021
@mergify mergify bot merged commit de57fa1 into ceph:devel Oct 27, 2021
@humblec
Copy link
Collaborator Author

humblec commented Oct 27, 2021

@Madhu-1 removed DNM on your behalf 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/rbd Issues related to RBD priority-1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants