Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚠ pkg/webhook/admission: use Result.Message instead of Result.Reason #1539

Merged

Conversation

bhcleek
Copy link
Contributor

@bhcleek bhcleek commented May 23, 2021

Use Result.Message instead of Result.Reason for the user supplied
portion. Reason is intended to be machine readable while Message is
intended to be human readable. While each is documented as being
informational only, the Is* family of functions in
k8s.io/apimachinery/pkg/api/errors rely on the StatusReason.

This change allows controllers to rely on
"k8s.io/apimachinery/pkg/api/errors".IsForbidden to deal with errors
consistently regardless of whether the operation failed due to an
admission webhook implemented with controller-runtime, a standard
kubernetes failure (e.g related to RBAC), or another controller not
implemented with controller-runtime.

See kubernetes/kubernetes#101926 for more information

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 23, 2021
@k8s-ci-robot
Copy link
Contributor

Hi @bhcleek. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label May 23, 2021
@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 23, 2021
@bhcleek bhcleek changed the title ⚠️ pkg/webhook/admission: use Result.Message instead of Result.Reason (:warning:, major) pkg/webhook/admission: use Result.Message instead of Result.Reason May 23, 2021
@bhcleek bhcleek changed the title (:warning:, major) pkg/webhook/admission: use Result.Message instead of Result.Reason ⚠ (:warning:, major) pkg/webhook/admission: use Result.Message instead of Result.Reason May 23, 2021
@bhcleek bhcleek changed the title ⚠ (:warning:, major) pkg/webhook/admission: use Result.Message instead of Result.Reason ⚠ pkg/webhook/admission: use Result.Message instead of Result.Reason May 23, 2021
@coderanger
Copy link
Contributor

Trying to think of ways this compat break could bite someone. We should be very certain we can't think of any.

@bhcleek
Copy link
Contributor Author

bhcleek commented May 23, 2021

Trying to think of ways this compat break could bite someone. We should be very certain we can't think of any.

I was torn on whether to mark this with ⚠️ or not, but decided to err on the conservative side.

I don't think this will break any controller-runtime clients, but it will cause "k8s.io/apimachinery/pkg/errors".IsForbidden calls to match Deny responses where previously it would not. This is actually a good thing, because controller code often checks IsForbidden (and other functions in the Is* family) to decide whether to retry an operation or not.

@bhcleek
Copy link
Contributor Author

bhcleek commented Jun 30, 2021

Is there anything we can do to move this forward?

@bowei
Copy link

bowei commented Jul 11, 2021

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 11, 2021
@bhcleek
Copy link
Contributor Author

bhcleek commented Jul 11, 2021

/retest

Use Result.Message instead of Result.Reason for the user supplied
portion. Reason is intended to be machine readable while Message is
intended to be human readable. While each is documented as being
informational only, the Is* family of functions in
k8s.io/apimachinery/pkg/api/errors rely on the StatusReason.

This change allows controllers to rely on
"k8s.io/apimachinery/pkg/api/errors".IsForbidden to deal with errors
consistently regardless of whether the operation failed due to an
admission webhook implemented with controller-runtime, a standard
kubernetes failure (e.g related to RBAC), or another controller not
implemented with controller-runtime.

See kubernetes/kubernetes#101926 for more information
@bhcleek bhcleek force-pushed the message-human-readable branch from 3ff152d to 2c95a03 Compare July 14, 2021 04:41
Copy link
Member

@joelanford joelanford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the piece of this that gives me some pause is that this change is API compatible, but it breaks semantics. So calling code will compile and run just fine, but what's happening may not be what a caller expects (e.g. perhaps a caller really does want to set a custom reason)

I do agree that it makes more sense to expose the message and auto-populate the reason.

I would almost prefer that we find a way to break the Go API so that users are confronted with this breakage.

},
},
}
if len(reason) > 0 {
resp.Result.Reason = metav1.StatusReason(reason)
if len(message) > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Do we need to check the length of message or can we just always set resp.Result.Message? metav1.Status.Message is a string type, so leaving it "unset" is the same as setting it to an empty string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, we no longer need to check the length here. Is it worth changing or should we keep it as-is if there aren't other changes to be made?

Copy link
Contributor Author

@bhcleek bhcleek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the piece of this that gives me some pause is that this change is API compatible, but it breaks semantics. So calling code will compile and run just fine, but what's happening may not be what a caller expects (e.g. perhaps a caller really does want to set a custom reason)

I do agree that it makes more sense to expose the message and auto-populate the reason.

I would almost prefer that we find a way to break the Go API so that users are confronted with this breakage.

Given that controller-runtime is still pre-1.0, this kind of breaking change in semantics seems appropriate to me, but I'll happily defer to you if you feel strongly. I see a couple of possibilities for addressing your concerns about the changing semantics:

  1. break the API by removing the second parameter of ValidationResponse entirely so that existing code will fail on compile and introduce a new ValidationResponseWithReasonAndMessage to allow both the reason and message to be provided.
  2. Keep API compatibility, but introduce func ValidationResponseWithReason(allowed bool, reason, message string) Response to allow users to migrate if they really do want to provide a custom reason.

What do you think?

},
},
}
if len(reason) > 0 {
resp.Result.Reason = metav1.StatusReason(reason)
if len(message) > 0 {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, we no longer need to check the length here. Is it worth changing or should we keep it as-is if there aren't other changes to be made?

@alvaroaleman
Copy link
Member

@bhcleek is this still needed now that kubernetes/kubernetes#101926 merged? Like Joe I'd prefer not to introduce breaking changes the compiler can't point out.

@bhcleek
Copy link
Contributor Author

bhcleek commented Aug 16, 2021

While this isn't strictly needed with the next version of Kubernetes or later, it's still useful for people on the current version of Kubernetes or earlier.

I'd also add, that IMHO this is closer to the expectations that people may have given how this message can be surfaced to users.

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 15, 2021
@bhcleek
Copy link
Contributor Author

bhcleek commented Nov 15, 2021

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 15, 2021
@bhcleek
Copy link
Contributor Author

bhcleek commented Feb 2, 2022

Where do we stand with this? Should we close it or is there something we can do to push forward on the alignment with kubernetes error expectations?

Copy link
Member

@alvaroaleman alvaroaleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/hold
@joelanford are you good with this?

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 5, 2022
@bhcleek
Copy link
Contributor Author

bhcleek commented Jun 1, 2022

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 1, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 30, 2022
@bhcleek
Copy link
Contributor Author

bhcleek commented Sep 3, 2022

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 3, 2022
@sbueringer
Copy link
Member

sbueringer commented Sep 5, 2022

I think it would be good if this change is surfaced via compile errors to consumers.

I would suggest:

  • Allowed() Response
    • Doesn't need any reason or message.
    • If folks want to set them anyway for an allowed request they can use ValidationResponse, but this shouldn't be neccessary:
      // Result contains extra details into why an admission request was denied.
      // This field IS NOT consulted in any way if "Allowed" is "true".
      // +optional
      Result *metav1.Status `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
  • Denied() Response
    • Essentially a shortcut to return deny with metav1.StatusReasonForbidden set
  • Patched(patches ...jsonpatch.JsonPatchOperation) Response
    • Doesn't need reason/message as it's also an allowed request (see above)
  • ValidationResponse(allowed bool, reason, message string) Response
    • Just one simple func where everything can be provided
    • If reason is "" and allowed is false I would set reason to metav1.StatusReasonForbidden

I think this would signal clearly to consumers that the API changed and the new set of funcs should cover relevant use cases.

@bhcleek
Copy link
Contributor Author

bhcleek commented Sep 5, 2022

I can make that change. The reason, though, should probably be a metav1.StatusReason instead of a string so that users cannot easily swap the reason and message arguments unintentionally. But what value should be used for the reason when allowed is set?

edit: formatting

@sbueringer
Copy link
Member

sbueringer commented Sep 5, 2022

I can make that change. The reason, though, should probably be a metav1.StatusReason instead of a string so that users cannot easily swap the reason and message arguments unintentionally. But what value should be used for the reason when allowed is set?

Sounds good. I don't think we have to set the reason when allowed is true. reason is part of Result and Result " contains extra details into why an admission request was denied". So I think if the request is allowed we don't have to provide details about that.

(btw just my opinion, would be probably good to get consensus before investing the effort to change the implementation)

@bhcleek
Copy link
Contributor Author

bhcleek commented Sep 5, 2022

Yeah, the few places I need it, metav1.StatusReasonUnknown can be used for success (e.g. Allowed())

@sbueringer
Copy link
Member

Yeah, the few places I need it, metav1.StatusReasonUnknown can be used for success (e.g. Allowed())

Why is a reason needed for Allowed?

@bhcleek
Copy link
Contributor Author

bhcleek commented Sep 5, 2022

If ValidationResponse is to have three parameters as you suggested with ValidationResponse(allowed bool, reason, message string), then an argument for the reason has to be provided.

@bhcleek
Copy link
Contributor Author

bhcleek commented Sep 5, 2022

After making the changes locally that you suggested, @sbueringer, I'm less convinced that's the direction this should go. Being able to provide the human readable message to Allowed and Denied is useful and requiring users provide the reason to ValidationResponse puts more burden on users to understand all the possible StatusReason values.

I know you're trying to avoid unexpected breakages, but given that controller-runtime regularly breaks across releases and effectively abuses semantic versioning anyway in order to be able to break backward compatibility without violating semantic versioning coupled with the greater burden that those suggestions would put on users, I'm not sure we should make those changes. 🤔

Having expressed my skepticism, I want to be clear that I will submit the changes if there's consensus.

@sbueringer
Copy link
Member

Okay no worries :). Just my opinion, I'm fine either way.

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 4, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jan 3, 2023
@bhcleek
Copy link
Contributor Author

bhcleek commented Jan 3, 2023

/remove-lifecycle rotten

@alvaroaleman with only 21 open PRs, I'm wondering if there's anything we can do to move this forward.

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jan 3, 2023
@alvaroaleman
Copy link
Member

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 3, 2023
@k8s-ci-robot k8s-ci-robot merged commit 056869c into kubernetes-sigs:master Jan 3, 2023
@bhcleek bhcleek deleted the message-human-readable branch January 3, 2023 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants