From c1f8dd3a48172aaa779508918448392279243e1b Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Wed, 14 Sep 2022 14:27:33 -0700
Subject: [PATCH 01/23] add kubectl explain: openapi v3 upgrade kep

---
 .../3515-kubectl-explain-openapiv3/README.md  | 862 ++++++++++++++++++
 .../3515-kubectl-explain-openapiv3/kep.yaml   |  23 +
 2 files changed, 885 insertions(+)
 create mode 100644 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
 create mode 100755 keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
new file mode 100644
index 00000000000..848a9f40b41
--- /dev/null
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -0,0 +1,862 @@
+<!--
+**Note:** When your KEP is complete, all of these comment blocks should be removed.
+
+To get started with this template:
+
+- [ ] **Pick a hosting SIG.**
+  Make sure that the problem space is something the SIG is interested in taking
+  up. KEPs should not be checked in without a sponsoring SIG.
+- [ ] **Create an issue in kubernetes/enhancements**
+  When filing an enhancement tracking issue, please make sure to complete all
+  fields in that template. One of the fields asks for a link to the KEP. You
+  can leave that blank until this KEP is filed, and then go back to the
+  enhancement and add the link.
+- [ ] **Make a copy of this template directory.**
+  Copy this template into the owning SIG's directory and name it
+  `NNNN-short-descriptive-title`, where `NNNN` is the issue number (with no
+  leading-zero padding) assigned to your enhancement above.
+- [ ] **Fill out as much of the kep.yaml file as you can.**
+  At minimum, you should fill in the "Title", "Authors", "Owning-sig",
+  "Status", and date-related fields.
+- [ ] **Fill out this file as best you can.**
+  At minimum, you should fill in the "Summary" and "Motivation" sections.
+  These should be easy if you've preflighted the idea of the KEP with the
+  appropriate SIG(s).
+- [ ] **Create a PR for this KEP.**
+  Assign it to people in the SIG who are sponsoring this process.
+- [ ] **Merge early and iterate.**
+  Avoid getting hung up on specific details and instead aim to get the goals of
+  the KEP clarified and merged quickly. The best way to do this is to just
+  start with the high-level sections and fill out details incrementally in
+  subsequent PRs.
+
+Just because a KEP is merged does not mean it is complete or approved. Any KEP
+marked as `provisional` is a working document and subject to change. You can
+denote sections that are under active debate as follows:
+
+```
+<<[UNRESOLVED optional short context or usernames ]>>
+Stuff that is being argued.
+<<[/UNRESOLVED]>>
+```
+
+When editing KEPS, aim for tightly-scoped, single-topic PRs to keep discussions
+focused. If you disagree with what is already in a document, open a new PR
+with suggested changes.
+
+One KEP corresponds to one "feature" or "enhancement" for its whole lifecycle.
+You do not need a new KEP to move from beta to GA, for example. If
+new details emerge that belong in the KEP, edit the KEP. Once a feature has become
+"implemented", major changes should get new KEPs.
+
+The canonical place for the latest set of instructions (and the likely source
+of this file) is [here](/keps/NNNN-kep-template/README.md).
+
+**Note:** Any PRs to move a KEP to `implementable`, or significant changes once
+it is marked `implementable`, must be approved by each of the KEP approvers.
+If none of those approvers are still appropriate, then changes to that list
+should be approved by the remaining approvers and/or the owning SIG (or
+SIG Architecture for cross-cutting KEPs).
+-->
+# KEP-3515: OpenAPI v3 for kubectl explain
+
+<!--
+This is the title of your KEP. Keep it short, simple, and descriptive. A good
+title can help communicate what the KEP is and should be considered as part of
+any review.
+-->
+
+<!--
+A table of contents is helpful for quickly jumping to sections of a KEP and for
+highlighting any additional information provided beyond the standard KEP
+template.
+
+Ensure the TOC is wrapped with
+  <code>&lt;!-- toc --&rt;&lt;!-- /toc --&rt;</code>
+tags, and then generate with `hack/update-toc.sh`.
+-->
+
+<!-- toc -->
+- [Release Signoff Checklist](#release-signoff-checklist)
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Richer OpenAPI V3 Data](#richer-openapi-v3-data)
+  - [CRD schemas expressed as OpenAPI v2 are lossy](#crd-schemas-expressed-as-openapi-v2-are-lossy)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [Basic Usage](#basic-usage)
+  - [Built-in Template Options](#built-in-template-options)
+  - [Custom Templates](#custom-templates)
+    - [Example template](#example-template)
+  - [Risks and Mitigations](#risks-and-mitigations)
+    - [OpenAPI V3 Not Available](#openapi-v3-not-available)
+      - [Risk](#risk)
+      - [Mitigation](#mitigation)
+    - [Template API Stability](#template-api-stability)
+      - [Risk](#risk-1)
+      - [Mitigation](#mitigation-1)
+    - [OpenAPI serialization time](#openapi-serialization-time)
+      - [Risk](#risk-2)
+      - [Mitigation](#mitigation-2)
+- [Design Details](#design-details)
+    - [Current High-level Approach](#current-high-level-approach)
+    - [Proposed High-level Approach](#proposed-high-level-approach)
+  - [Template rendering](#template-rendering)
+  - [Test Plan](#test-plan)
+      - [Prerequisite testing updates](#prerequisite-testing-updates)
+      - [Unit tests](#unit-tests)
+      - [Integration tests](#integration-tests)
+      - [e2e tests](#e2e-tests)
+  - [Graduation Criteria](#graduation-criteria)
+    - [Alpha](#alpha)
+    - [Beta](#beta)
+    - [GA](#ga)
+  - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
+  - [Version Skew Strategy](#version-skew-strategy)
+- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
+  - [Feature Enablement and Rollback](#feature-enablement-and-rollback)
+  - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning)
+  - [Monitoring Requirements](#monitoring-requirements)
+  - [Dependencies](#dependencies)
+  - [Scalability](#scalability)
+  - [Troubleshooting](#troubleshooting)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+  - [Implement proto.Models for OpenAPI V3 data](#implement-protomodels-for-openapi-v3-data)
+<!-- /toc -->
+
+## Release Signoff Checklist
+
+<!--
+**ACTION REQUIRED:** In order to merge code into a release, there must be an
+issue in [kubernetes/enhancements] referencing this KEP and targeting a release
+milestone **before the [Enhancement Freeze](https://git.k8s.io/sig-release/releases)
+of the targeted release**.
+
+For enhancements that make changes to code or processes/procedures in core
+Kubernetes—i.e., [kubernetes/kubernetes], we require the following Release
+Signoff checklist to be completed.
+
+Check these off as they are completed for the Release Team to track. These
+checklist items _must_ be updated for the enhancement to be released.
+-->
+
+Items marked with (R) are required *prior to targeting to a milestone / release*.
+
+- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
+- [ ] (R) KEP approvers have approved the KEP status as `implementable`
+- [ ] (R) Design details are appropriately documented
+- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
+  - [ ] e2e Tests for all Beta API Operations (endpoints)
+  - [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
+  - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
+- [ ] (R) Graduation criteria is in place
+  - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
+- [ ] (R) Production readiness review completed
+- [ ] (R) Production readiness review approved
+- [ ] "Implementation History" section is up-to-date for milestone
+- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
+- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
+
+<!--
+**Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone.
+-->
+
+[kubernetes.io]: https://kubernetes.io/
+[kubernetes/enhancements]: https://git.k8s.io/enhancements
+[kubernetes/kubernetes]: https://git.k8s.io/kubernetes
+[kubernetes/website]: https://git.k8s.io/website
+
+## Summary
+
+This KEP proposes an enhancement to kubectl explain:
+
+1. Switch data source from OpenAPI v2 to OpenAPI v3
+2. Replace the hand-written `kubectl explain` printer with a go/template implementation.
+
+## Motivation
+
+<!--
+This section is for explicitly listing the motivation, goals, and non-goals of
+this KEP.  Describe why the change is important and the benefits to users. The
+motivation section can optionally provide links to [experience reports] to
+demonstrate the interest in a KEP within the wider Kubernetes community.
+
+[experience reports]: https://github.com/golang/go/wiki/ExperienceReports
+-->
+
+### Richer OpenAPI V3 Data
+
+OpenAPI V3 support in Kubernetes is planned to go GA in 1.26. OpenAPI V3 is a richer
+representation of the kubernetes API to our users, who have been asking for visibility
+into things like:
+
+1. nullable
+2. default
+2. validation fields like oneOf, anyOf
+2. <jeffrey what else does openapi v3 add>
+
+### CRD schemas expressed as OpenAPI v2 are lossy
+
+Today CRDs specify their schemas in OpenAPI v3 format. To serve the `/openapi/v2`
+document used today by kubectl, there is an expensive conversion from the v3 down
+to v2 format.
+
+This process is [very lossy](https://github.com/kubernetes/kubernetes/blob/6e0de20fbb4c127d2e45c7a22347c08545fc7a86/staging/src/k8s.io/apiextensions-apiserver/pkg/controller/openapi/v2/conversion.go#L56-L66), so `kubectl explain` when used against CRDs
+making use of v3 features does not have a good experience with inaccurate information, or fields removed altogther.
+
+
+### Goals
+
+1. Provide the new richer type information specified by OpenAPI v3 within kubectl explain
+2. Have a more maintainable `text/template` based approach to printing
+3. Fallback to old `explain` implementation if cluster does not expose OpenAPI v3 data.
+4. Provide multiple new output formats for kubectl explain:
+    * human-readable plaintext
+    * markdown
+    * maybe others
+5. (Optional?) Allow users to specify their own templates for use with kubectl
+  explain
+
+### Non-Goals
+
+any?
+
+## Proposal
+
+
+### Basic Usage
+
+The following user experience should be possible with `kubectl explain`
+
+```shell
+kubectl explain pods
+```
+
+Output should be familiar to users of today's `kubectl explain`, except new
+information is populated:
+
+```console
+<TODO>
+```
+
+### Built-in Template Options
+
+Users should be able to specify built-in templates to use instead of the 
+human-readable plaintext form:
+```shell
+kubectl explain pods --template md
+```
+
+When using the `md` template, a markdown document is printed to stdout, so it
+might be saved and used for a documentation website, for example.
+
+```console
+<TODO>
+```
+
+### Custom Templates
+Users should also be able to specify a path to a custom template file for
+the resource information to be written to:
+
+human-readable plaintext form:
+```shell
+kubectl explain pods --template /path/to/template.tmpl
+```
+
+#### Example template
+
+TBD
+
+### Risks and Mitigations
+
+#### OpenAPI V3 Not Available
+
+##### Risk
+
+OpenAPI v3 data is not available in the current cluster.
+
+##### Mitigation
+
+If the initial request for OpenAPI V3 fails, the old implementation using
+OpenAPI v2 should be used instead. There is another risk that this fallback process
+takes too long for interactive speed, so timeout should be carefully considered
+to prevent user from waiting too long to see their results. 
+
+#### Template API Stability
+##### Risk
+
+If templates are written to target a specific OpenAPI version, then changes to
+the format would case templates to be broken. It is a goal of this KEP to be more
+maintainable than the current verison. If we risk templates breaking due to format
+changes, that voids the goals of the KEP.
+
+This is only a risk of users are allowed to specify their own templates. If all
+templates are maintained internally, then API stability it is something to consider
+but not necessarily a requirement.
+
+##### Mitigation
+
+TBD
+
+#### OpenAPI serialization time
+##### Risk
+
+Today there is no interactive-speed way to deserialize protobuf or JSON openapi
+v3 data into the kube-openapi format.
+
+##### Mitigation
+
+There has been recent progress in this area. To unmarshal kube-OpenAPI v3 is now able
+to be done in a performant enough way to do it in the CLI. This KEP's beta release
+should be blocked on the merging of this optimization.
+
+## Design Details
+
+#### Current High-level Approach
+
+TODO: fill with links to referred implementation
+
+1. User types `kubectl explain pods`
+2. kubectl resolves 'pods' to GVR core v1 pods using cluster discovery information
+3. kubectl resolves GVR to its GVK using restmapper
+4. kubectl fetches `/openapi/v2` as protobuf
+5. kubectl parses the protobuf into `gnostic_v2.Document`
+6. kubectl converts `gnostic_v2.Document` into `proto.Models`
+7. kubectl searches the document's `Definitions` for a schema with the
+extension `x-kubernetes-group-version-kind` matching the interested GVK
+8. kubectl renders the definition using its hardcoded printer
+9. If `--recursive` was used, repeat step 8 for the transitive closure of
+  object-typed fields of the top-level object. Concat the results together.
+
+#### Proposed High-level Approach
+
+1. User types `kubectl explain pods`
+2. kubectl resolves 'pods' to GVR core v1 pods using cluster discovery information
+3. kubectl fetches `/openapi/v3/<group>/<version>`
+4. kubectl parses the result as kube-openapi spec3
+5. kubectl locates the return type of the `/apis/<group>/<version>/<resource>` OpenAPI Path
+6. kubectl renders the type using its built-in template for human-readable plaintext
+7. If `--recursive` was used, repeat step 6 for the transitive closure of
+  object-typed fields of the top-level object. Concat the results together.
+
+### Template rendering
+
+Go's text/template will be used due to its familiarity, stability, and virtue of being in stdlib.
+
+
+The following library functions will be provided to templates:
+
+TBD
+
+### Test Plan
+
+<!--
+**Note:** *Not required until targeted at a release.*
+The goal is to ensure that we don't accept enhancements with inadequate testing.
+
+All code is expected to have adequate tests (eventually with coverage
+expectations). Please adhere to the [Kubernetes testing guidelines][testing-guidelines]
+when drafting this test plan.
+
+[testing-guidelines]: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
+-->
+
+[ ] I/we understand the owners of the involved components may require updates to
+existing tests to make this code solid enough prior to committing the changes necessary
+to implement this enhancement.
+
+##### Prerequisite testing updates
+
+<!--
+Based on reviewers feedback describe what additional tests need to be added prior
+implementing this enhancement to ensure the enhancements have also solid foundations.
+-->
+
+##### Unit tests
+
+<!--
+In principle every added code should have complete unit test coverage, so providing
+the exact set of tests will not bring additional value.
+However, if complete unit test coverage is not possible, explain the reason of it
+together with explanation why this is acceptable.
+-->
+
+<!--
+Additionally, for Alpha try to enumerate the core package you will be touching
+to implement this enhancement and provide the current unit coverage for those
+in the form of:
+- <package>: <date> - <current test coverage>
+The data can be easily read from:
+https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-coverage-unit
+
+This can inform certain test coverage improvements that we want to do before
+extending the production code to implement this enhancement.
+-->
+
+- `<package>`: `<date>` - `<test coverage>`
+
+##### Integration tests
+
+<!--
+This question should be filled when targeting a release.
+For Alpha, describe what tests will be added to ensure proper quality of the enhancement.
+
+For Beta and GA, add links to added tests together with links to k8s-triage for those tests:
+https://storage.googleapis.com/k8s-triage/index.html
+-->
+
+- <test>: <link to test coverage>
+
+##### e2e tests
+
+<!--
+This question should be filled when targeting a release.
+For Alpha, describe what tests will be added to ensure proper quality of the enhancement.
+
+For Beta and GA, add links to added tests together with links to k8s-triage for those tests:
+https://storage.googleapis.com/k8s-triage/index.html
+
+We expect no non-infra related flakes in the last month as a GA graduation criteria.
+-->
+
+- <test>: <link to test coverage>
+
+### Graduation Criteria
+
+Defined using feature gate
+
+#### Alpha
+
+- Feature implemented behind an envar?
+- Existing explain tests are working or adapted for new implementation
+
+#### Beta
+
+- kube-OpenAPI JSON deserialization is optimized to take less than 150ms on 
+  most machines
+
+#### GA
+
+<!--
+**Note:** *Not required until targeted at a release.*
+
+Define graduation milestones.
+
+These may be defined in terms of API maturity, [feature gate] graduations, or as
+something else. The KEP should keep this high-level with a focus on what
+signals will be looked at to determine graduation.
+
+Consider the following in developing the graduation criteria for this enhancement:
+- [Maturity levels (`alpha`, `beta`, `stable`)][maturity-levels]
+- [Feature gate][feature gate] lifecycle
+- [Deprecation policy][deprecation-policy]
+
+Clearly define what graduation means by either linking to the [API doc
+definition](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning)
+or by redefining what graduation means.
+
+In general we try to use the same stages (alpha, beta, GA), regardless of how the
+functionality is accessed.
+
+[feature gate]: https://git.k8s.io/community/contributors/devel/sig-architecture/feature-gates.md
+[maturity-levels]: https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions
+[deprecation-policy]: https://kubernetes.io/docs/reference/using-api/deprecation-policy/
+
+Below are some examples to consider, in addition to the aforementioned [maturity levels][maturity-levels].
+
+#### Alpha
+
+- Feature implemented behind a feature flag
+- Initial e2e tests completed and enabled
+
+#### Beta
+
+- Gather feedback from developers and surveys
+- Complete features A, B, C
+- Additional tests are in Testgrid and linked in KEP
+
+#### GA
+
+- N examples of real-world usage
+- N installs
+- More rigorous forms of testing—e.g., downgrade tests and scalability tests
+- Allowing time for feedback
+
+**Note:** Generally we also wait at least two releases between beta and
+GA/stable, because there's no opportunity for user feedback, or even bug reports,
+in back-to-back releases.
+
+**For non-optional features moving to GA, the graduation criteria must include
+[conformance tests].**
+
+[conformance tests]: https://git.k8s.io/community/contributors/devel/sig-architecture/conformance-tests.md
+
+#### Deprecation
+
+- Announce deprecation and support policy of the existing flag
+- Two versions passed since introducing the functionality that deprecates the flag (to address version skew)
+- Address feedback on usage/changed behavior, provided on GitHub issues
+- Deprecate the flag
+-->
+
+### Upgrade / Downgrade Strategy
+
+<!--
+If applicable, how will the component be upgraded and downgraded? Make sure
+this is in the test plan.
+
+Consider the following in developing an upgrade/downgrade strategy for this
+enhancement:
+- What changes (in invocations, configurations, API use, etc.) is an existing
+  cluster required to make on upgrade, in order to maintain previous behavior?
+- What changes (in invocations, configurations, API use, etc.) is an existing
+  cluster required to make on upgrade, in order to make use of the enhancement?
+-->
+
+### Version Skew Strategy
+
+<!--
+If applicable, how will the component handle version skew with other
+components? What are the guarantees? Make sure this is in the test plan.
+
+Consider the following in developing a version skew strategy for this
+enhancement:
+- Does this enhancement involve coordinating behavior in the control plane and
+  in the kubelet? How does an n-2 kubelet without this feature available behave
+  when this feature is used?
+- Will any other components on the node change? For example, changes to CSI,
+  CRI or CNI may require updating that component before the kubelet.
+-->
+
+## Production Readiness Review Questionnaire
+
+<!--
+
+Production readiness reviews are intended to ensure that features merging into
+Kubernetes are observable, scalable and supportable; can be safely operated in
+production environments, and can be disabled or rolled back in the event they
+cause increased failures in production. See more in the PRR KEP at
+https://git.k8s.io/enhancements/keps/sig-architecture/1194-prod-readiness.
+
+The production readiness review questionnaire must be completed and approved
+for the KEP to move to `implementable` status and be included in the release.
+
+In some cases, the questions below should also have answers in `kep.yaml`. This
+is to enable automation to verify the presence of the review, and to reduce review
+burden and latency.
+
+The KEP must have a approver from the
+[`prod-readiness-approvers`](http://git.k8s.io/enhancements/OWNERS_ALIASES)
+team. Please reach out on the
+[#prod-readiness](https://kubernetes.slack.com/archives/CPNHUMN74) channel if
+you need any help or guidance.
+-->
+
+### Feature Enablement and Rollback
+
+<!--
+This section must be completed when targeting alpha to a release.
+-->
+
+###### How can this feature be enabled / disabled in a live cluster?
+
+<!--
+Pick one of these and delete the rest.
+
+Documentation is available on [feature gate lifecycle] and expectations, as
+well as the [existing list] of feature gates.
+
+[feature gate lifecycle]: https://git.k8s.io/community/contributors/devel/sig-architecture/feature-gates.md
+[existing list]: https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/
+-->
+
+- [ ] Feature gate (also fill in values in `kep.yaml`)
+  - Feature gate name:
+  - Components depending on the feature gate:
+- [ ] Other
+  - Describe the mechanism:
+  - Will enabling / disabling the feature require downtime of the control
+    plane?
+  - Will enabling / disabling the feature require downtime or reprovisioning
+    of a node? (Do not assume `Dynamic Kubelet Config` feature is enabled).
+
+###### Does enabling the feature change any default behavior?
+
+<!--
+Any change of default behavior may be surprising to users or break existing
+automations, so be extremely careful here.
+-->
+
+###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
+
+Yes. The old hand-written printer for OpenAPI v2 data in use today can be used as
+a fallback.
+
+###### What happens if we reenable the feature if it was previously rolled back?
+
+Exactly what you would expect. Since the feature only changes how data is displayed,
+there is no lasting effect from toggling it.
+
+###### Are there any tests for feature enablement/disablement?
+
+<!--
+The e2e framework does not currently support enabling or disabling feature
+gates. However, unit tests in each component dealing with managing data, created
+with and without the feature, are necessary. At the very least, think about
+conversion tests if API types are being modified.
+
+Additionally, for features that are introducing a new API field, unit tests that
+are exercising the `switch` of feature gate itself (what happens if I disable a
+feature gate after having objects written with the new field) are also critical.
+You can take a look at one potential example of such test in:
+https://github.com/kubernetes/kubernetes/pull/97058/files#diff-7826f7adbc1996a05ab52e3f5f02429e94b68ce6bce0dc534d1be636154fded3R246-R282
+-->
+
+### Rollout, Upgrade and Rollback Planning
+
+<!--
+This section must be completed when targeting beta to a release.
+-->
+
+###### How can a rollout or rollback fail? Can it impact already running workloads?
+
+<!--
+Try to be as paranoid as possible - e.g., what if some components will restart
+mid-rollout?
+
+Be sure to consider highly-available clusters, where, for example,
+feature flags will be enabled on some API servers and not others during the
+rollout. Similarly, consider large clusters and how enablement/disablement
+will rollout across nodes.
+-->
+
+###### What specific metrics should inform a rollback?
+
+<!--
+What signals should users be paying attention to when the feature is young
+that might indicate a serious problem?
+-->
+
+###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?
+
+<!--
+Describe manual testing that was done and the outcomes.
+Longer term, we may want to require automated upgrade/rollback tests, but we
+are missing a bunch of machinery and tooling and can't do that now.
+-->
+
+###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.?
+
+<!--
+Even if applying deprecation policies, they may still surprise some users.
+-->
+
+### Monitoring Requirements
+
+<!--
+This section must be completed when targeting beta to a release.
+
+For GA, this section is required: approvers should be able to confirm the
+previous answers based on experience in the field.
+-->
+
+###### How can an operator determine if the feature is in use by workloads?
+
+<!--
+Ideally, this should be a metric. Operations against the Kubernetes API (e.g.,
+checking if there are objects with field X set) may be a last resort. Avoid
+logs or events for this purpose.
+-->
+
+###### How can someone using this feature know that it is working for their instance?
+
+```shell
+kubectl explain pods -v
+```
+
+TBD exact output. Should see successful OpenAPI v3 request or something
+
+###### What are the reasonable SLOs (Service Level Objectives) for the enhancement?
+
+<!--
+This is your opportunity to define what "normal" quality of service looks like
+for a feature.
+
+It's impossible to provide comprehensive guidance, but at the very
+high level (needs more precise definitions) those may be things like:
+  - per-day percentage of API calls finishing with 5XX errors <= 1%
+  - 99% percentile over day of absolute value from (job creation time minus expected
+    job creation time) for cron job <= 10%
+  - 99.9% of /health requests per day finish with 200 code
+
+These goals will help you determine what you need to measure (SLIs) in the next
+question.
+-->
+
+###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service?
+
+<!--
+Pick one more of these and delete the rest.
+-->
+
+- [ ] Metrics
+  - Metric name:
+  - [Optional] Aggregation method:
+  - Components exposing the metric:
+- [ ] Other (treat as last resort)
+  - Details:
+
+###### Are there any missing metrics that would be useful to have to improve observability of this feature?
+
+<!--
+Describe the metrics themselves and the reasons why they weren't added (e.g., cost,
+implementation difficulties, etc.).
+-->
+
+### Dependencies
+
+<!--
+This section must be completed when targeting beta to a release.
+-->
+
+###### Does this feature depend on any specific services running in the cluster?
+
+<!--
+Think about both cluster-level services (e.g. metrics-server) as well
+as node-level agents (e.g. specific version of CRI). Focus on external or
+optional services that are needed. For example, if this feature depends on
+a cloud provider API, or upon an external software-defined storage or network
+control plane.
+
+For each of these, fill in the following—thinking about running existing user workloads
+and creating new ones, as well as about cluster-level services (e.g. DNS):
+  - [Dependency name]
+    - Usage description:
+      - Impact of its outage on the feature:
+      - Impact of its degraded performance or high-error rates on the feature:
+-->
+
+To reap the benefits of this feature, OpenAPI v3 is required, however OpenAPI v2
+data can be used as a fallback.
+
+### Scalability
+
+<!--
+For alpha, this section is encouraged: reviewers should consider these questions
+and attempt to answer them.
+
+For beta, this section is required: reviewers must answer these questions.
+
+For GA, this section is required: approvers should be able to confirm the
+previous answers based on experience in the field.
+-->
+
+###### Will enabling / using this feature result in any new API calls?
+
+Yes, up feature replaces a single GET of `/openapi/v2` which returns a large (megabytes)
+openapi document for all types with a more targeted call to `/openapi/v3/<group>/<version>`
+
+The `/openapi/v3` endpoing implements E-Tag caching so that if the document has
+not changed the server incurs a cheap, almost neglible cost to serving the request.
+
+<!--
+Describe them, providing:
+  - API call type (e.g. PATCH pods)
+  - estimated throughput
+  - originating component(s) (e.g. Kubelet, Feature-X-controller)
+Focusing mostly on:
+  - components listing and/or watching resources they didn't before
+  - API calls that may be triggered by changes of some Kubernetes resources
+    (e.g. update of object X triggers new updates of object Y)
+  - periodic API calls to reconcile state (e.g. periodic fetching state,
+    heartbeats, leader election, etc.)
+-->
+
+###### Will enabling / using this feature result in introducing new API types?
+
+No.
+
+###### Will enabling / using this feature result in any new calls to the cloud provider?
+
+No.
+
+###### Will enabling / using this feature result in increasing size or count of the existing API objects?
+
+No.
+
+###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?
+
+No.
+
+###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components?
+
+No, would expect generally same amount of resource usage for kubectl.
+
+### Troubleshooting
+
+<!--
+This section must be completed when targeting beta to a release.
+
+For GA, this section is required: approvers should be able to confirm the
+previous answers based on experience in the field.
+
+The Troubleshooting section currently serves the `Playbook` role. We may consider
+splitting it into a dedicated `Playbook` document (potentially with some monitoring
+details). For now, we leave it here.
+-->
+
+###### How does this feature react if the API server and/or etcd is unavailable?
+
+
+
+###### What are other known failure modes?
+
+<!--
+For each of them, fill in the following information by copying the below template:
+  - [Failure mode brief description]
+    - Detection: How can it be detected via metrics? Stated another way:
+      how can an operator troubleshoot without logging into a master or worker node?
+    - Mitigations: What can be done to stop the bleeding, especially for already
+      running user workloads?
+    - Diagnostics: What are the useful log messages and their required logging
+      levels that could help debug the issue?
+      Not required until feature graduated to beta.
+    - Testing: Are there any tests for failure mode? If not, describe why.
+-->
+
+###### What steps should be taken if SLOs are not being met to determine the problem?
+
+## Implementation History
+
+<!--
+Major milestones in the lifecycle of a KEP should be tracked in this section.
+Major milestones might include:
+- the `Summary` and `Motivation` sections being merged, signaling SIG acceptance
+- the `Proposal` section being merged, signaling agreement on a proposed design
+- the date implementation started
+- the first Kubernetes release where an initial version of the KEP was available
+- the version of Kubernetes where the KEP graduated to general availability
+- when the KEP was retired or superseded
+-->
+
+## Drawbacks
+
+<!--
+Why should this KEP _not_ be implemented?
+-->
+
+## Alternatives
+
+### Implement proto.Models for OpenAPI V3 data
+
+The current hard-coded printer is capable of printing any objects in `proto.Models` form.
+
+[We already have a way to express OpenAPI v3 data as `proto.Models`, so this can be
+seen as a path of least resistence for plugging OpenAPI v3 into `kubectl explain`.
+
+We decide against this approach due to our desire to deprecate `proto.Models`.
+We see`proto.Models` conversion as unnecessary and costly buraucracy in the code,
+and we are seeking to deprecate the type in favor of the kube-openapi types.
diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
new file mode 100755
index 00000000000..27569805bb6
--- /dev/null
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
@@ -0,0 +1,23 @@
+id: "3515"
+prnumber: ""
+name: kubectl-explain-openapiv3
+title: Kubectl Explain OpenAPIv3
+kep-number: draft
+authors: ['@alexzielenski']
+owning-sig: sig-cli
+participating-sigs: [sig-api-machinery, sig-cli]
+reviewers: []
+approvers: ['@TBD']
+prr-approvers: []
+creation-date: "2022-09-14"
+last-updated: v1.19
+status: provisional
+stage: alpha
+latest-milestone: ""
+milestone:
+    alpha: ""
+    beta: ""
+    stable: ""
+feature-gates: []
+disable-supported: false
+metrics: []

From a5814d16e23eb817ba583bde6f1f76a9bfaee299 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Tue, 20 Sep 2022 16:09:43 -0700
Subject: [PATCH 02/23] spruce up wording

---
 .../3515-kubectl-explain-openapiv3/README.md  | 41 +++++++++++++------
 1 file changed, 29 insertions(+), 12 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 848a9f40b41..c88d801e544 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -187,16 +187,19 @@ demonstrate the interest in a KEP within the wider Kubernetes community.
 [experience reports]: https://github.com/golang/go/wiki/ExperienceReports
 -->
 
-### Richer OpenAPI V3 Data
+### OpenAPI v3 is a richer API description than OpenAPI v2
 
-OpenAPI V3 support in Kubernetes is planned to go GA in 1.26. OpenAPI V3 is a richer
-representation of the kubernetes API to our users, who have been asking for visibility
+OpenAPI v3 support in Kubernetes is currently beta since version 1.24.
+OpenAPI V3 is a richer representation of the kubernetes API to our users, who have been asking for visibility
 into things like:
 
 1. nullable
 2. default
-2. validation fields like oneOf, anyOf
-2. <jeffrey what else does openapi v3 add>
+3. validation fields like oneOf, anyOf, etc.
+
+To show each of these additional data points by themselves is a strong reason
+to switch to using OpenAPI v3.
+
 
 ### CRD schemas expressed as OpenAPI v2 are lossy
 
@@ -207,6 +210,9 @@ to v2 format.
 This process is [very lossy](https://github.com/kubernetes/kubernetes/blob/6e0de20fbb4c127d2e45c7a22347c08545fc7a86/staging/src/k8s.io/apiextensions-apiserver/pkg/controller/openapi/v2/conversion.go#L56-L66), so `kubectl explain` when used against CRDs
 making use of v3 features does not have a good experience with inaccurate information, or fields removed altogther.
 
+This transformation causes bugs, for example, when attempting to `explain` a field
+that is `nullable`, kubectl instead shows nothing, due to the lossy conversion
+wiping nullable fields.
 
 ### Goals
 
@@ -218,11 +224,16 @@ making use of v3 features does not have a good experience with inaccurate inform
     * markdown
     * maybe others
 5. (Optional?) Allow users to specify their own templates for use with kubectl
-  explain
+  explain (there may be interesting use cases for this)
 
 ### Non-Goals
 
-any?
+1. "Fix" openapi v3 to openapi v2 conversion
+  This is a non-goal for two reasons:
+    * These formats are not compatible, and there WILL be data loss and inaccuracy
+    * This negates the benefits of using OpenAPI v3 for the richer type information
+2. Provide general-purpose OpenAPI visualization.
+
 
 ## Proposal
 
@@ -258,7 +269,7 @@ might be saved and used for a documentation website, for example.
 ```
 
 ### Custom Templates
-Users should also be able to specify a path to a custom template file for
+Users might also like to be able to specify a path to a custom template file for
 the resource information to be written to:
 
 human-readable plaintext form:
@@ -266,7 +277,7 @@ human-readable plaintext form:
 kubectl explain pods --template /path/to/template.tmpl
 ```
 
-#### Example template
+#### Example template Format
 
 TBD
 
@@ -857,6 +868,12 @@ The current hard-coded printer is capable of printing any objects in `proto.Mode
 [We already have a way to express OpenAPI v3 data as `proto.Models`, so this can be
 seen as a path of least resistence for plugging OpenAPI v3 into `kubectl explain`.
 
-We decide against this approach due to our desire to deprecate `proto.Models`.
-We see`proto.Models` conversion as unnecessary and costly buraucracy in the code,
-and we are seeking to deprecate the type in favor of the kube-openapi types.
+This approach is undesirable for a few different reasons:
+
+1.) We would like to update the explain printer to include new OpenAPI v3 information,
+the current design makes that time consuming and not maintainable.
+
+2.) API-Machinery has desire to deprecate `proto.Models`. We see`proto.Models`
+conversion as unnecessary and costly buraucracy, that contributes to high
+OpenAPI overhead. We are seeking to deprecate the type in favor of the
+kube-openapi types for future usage.

From d03ed2131ad9b0cc6fdc5817c58f0a9b3d11cb84 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Wed, 28 Sep 2022 13:33:53 -0700
Subject: [PATCH 03/23] reduce scope and take feedback into account

---
 keps/prod-readiness/sig-cli/3515.yaml         |   3 +
 .../3515-kubectl-explain-openapiv3/README.md  | 120 +++++++++++-------
 .../3515-kubectl-explain-openapiv3/kep.yaml   |   4 +-
 3 files changed, 80 insertions(+), 47 deletions(-)
 create mode 100644 keps/prod-readiness/sig-cli/3515.yaml

diff --git a/keps/prod-readiness/sig-cli/3515.yaml b/keps/prod-readiness/sig-cli/3515.yaml
new file mode 100644
index 00000000000..b2bb14812b0
--- /dev/null
+++ b/keps/prod-readiness/sig-cli/3515.yaml
@@ -0,0 +1,3 @@
+kep-number: 3515
+alpha:
+  approver: "@johnbelamaric"
diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index c88d801e544..dcd4d28dab8 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -253,33 +253,63 @@ information is populated:
 <TODO>
 ```
 
+Note: Feature during development will be gated by an experimental flag. The commands
+shown here elide the experimental flag for clarity.
+
 ### Built-in Template Options
+#### Plaintext
+
 
-Users should be able to specify built-in templates to use instead of the 
-human-readable plaintext form:
 ```shell
-kubectl explain pods --template md
+kubectl explain pods
 ```
+or
 
-When using the `md` template, a markdown document is printed to stdout, so it
-might be saved and used for a documentation website, for example.
+```shell
+kubectl explain pods --output plaintext
+```
 
-```console
-<TODO>
+The plaintext output format is the default and should be crafted to be as close
+as the existing `explain` output in use before this KEP.
+
+#### Raw
+
+```shell
+kubectl explain pods --output raw
 ```
 
-### Custom Templates
-Users might also like to be able to specify a path to a custom template file for
-the resource information to be written to:
+To get raw OpenAPI v3 data for a certain resource today involves:
+1.) setting up kubectl proxy
+2.) fetching the correct path at `/openapi/v3/<group>/<version>`
+3.) filtering out unwanted results
+
+This command is useful not only for its convenience, but also other visualizations
+may be built upon the raw output if we opt not to support a first-class custom
+template solution in the future.
+
+#### HTML
 
-human-readable plaintext form:
 ```shell
-kubectl explain pods --template /path/to/template.tmpl
+kubectl explain pods --output html
 ```
 
-#### Example template Format
+Similarly to [godoc](https://pkg.go.dev), the we suggest to provide a searchable,
+navigable, generated webpage for the kubernetes types of whatever cluster kubectl
+is talking to.
+
+#### Markdown
+
+```shell
+kubectl explain pods --output md
+```
+
+When using the `md` template, a markdown document is printed to stdout, so it
+might be saved and used for a documentation website, for example.
+
+```console
+<TODO>
+```
 
-TBD
 
 ### Risks and Mitigations
 
@@ -296,21 +326,6 @@ OpenAPI v2 should be used instead. There is another risk that this fallback proc
 takes too long for interactive speed, so timeout should be carefully considered
 to prevent user from waiting too long to see their results. 
 
-#### Template API Stability
-##### Risk
-
-If templates are written to target a specific OpenAPI version, then changes to
-the format would case templates to be broken. It is a goal of this KEP to be more
-maintainable than the current verison. If we risk templates breaking due to format
-changes, that voids the goals of the KEP.
-
-This is only a risk of users are allowed to specify their own templates. If all
-templates are maintained internally, then API stability it is something to consider
-but not necessarily a requirement.
-
-##### Mitigation
-
-TBD
 
 #### OpenAPI serialization time
 ##### Risk
@@ -357,11 +372,6 @@ extension `x-kubernetes-group-version-kind` matching the interested GVK
 
 Go's text/template will be used due to its familiarity, stability, and virtue of being in stdlib.
 
-
-The following library functions will be provided to templates:
-
-TBD
-
 ### Test Plan
 
 <!--
@@ -441,12 +451,12 @@ Defined using feature gate
 
 #### Alpha
 
-- Feature implemented behind an envar?
+- Feature implemented behind a command line flag `--experimental-openapiv3`
 - Existing explain tests are working or adapted for new implementation
 
 #### Beta
 
-- kube-OpenAPI JSON deserialization is optimized to take less than 150ms on 
+- kube-OpenAPI v3 JSON deserialization is optimized to take less than 150ms on 
   most machines
 
 #### GA
@@ -587,12 +597,12 @@ well as the [existing list] of feature gates.
 - [ ] Feature gate (also fill in values in `kep.yaml`)
   - Feature gate name:
   - Components depending on the feature gate:
-- [ ] Other
-  - Describe the mechanism:
+- [x] Other
+  - Describe the mechanism: --experimental-openapiv3 flag
   - Will enabling / disabling the feature require downtime of the control
-    plane?
+    plane? No
   - Will enabling / disabling the feature require downtime or reprovisioning
-    of a node? (Do not assume `Dynamic Kubelet Config` feature is enabled).
+    of a node? (Do not assume `Dynamic Kubelet Config` feature is enabled). No
 
 ###### Does enabling the feature change any default behavior?
 
@@ -606,10 +616,12 @@ automations, so be extremely careful here.
 Yes. The old hand-written printer for OpenAPI v2 data in use today can be used as
 a fallback.
 
+Until the feature is stable it will only be enabled when the `--experimental-openapiv3` flag is used.
+
 ###### What happens if we reenable the feature if it was previously rolled back?
 
-Exactly what you would expect. Since the feature only changes how data is displayed,
-there is no lasting effect from toggling it.
+There is no persistence to using the `--experimental-openapiv3` flag. It is only
+used for viewing data.
 
 ###### Are there any tests for feature enablement/disablement?
 
@@ -685,10 +697,10 @@ logs or events for this purpose.
 ###### How can someone using this feature know that it is working for their instance?
 
 ```shell
-kubectl explain pods -v
+kubectl explain pods --output raw
 ```
 
-TBD exact output. Should see successful OpenAPI v3 request or something
+TBD exact output. User should see OpenAPI v3 JSON Schema for `pods` type printed to console.
 
 ###### What are the reasonable SLOs (Service Level Objectives) for the enhancement?
 
@@ -770,9 +782,13 @@ previous answers based on experience in the field.
 Yes, up feature replaces a single GET of `/openapi/v2` which returns a large (megabytes)
 openapi document for all types with a more targeted call to `/openapi/v3/<group>/<version>`
 
-The `/openapi/v3` endpoing implements E-Tag caching so that if the document has
+The `/openapi/v3/<group>/<version>` endpoint implements E-Tag caching so that if the document has
 not changed the server incurs a cheap, almost neglible cost to serving the request.
 
+The document returned by calls to `/openapi/v3/...` is expected to be far smaller
+than the megabytes-scale openapi v2 document, since it only includes information
+for a single group-version.
+
 <!--
 Describe them, providing:
   - API call type (e.g. PATCH pods)
@@ -877,3 +893,17 @@ the current design makes that time consuming and not maintainable.
 conversion as unnecessary and costly buraucracy, that contributes to high
 OpenAPI overhead. We are seeking to deprecate the type in favor of the
 kube-openapi types for future usage.
+
+### Custom User Templates
+Users might also like to be able to specify a path to a custom template file for
+the resource information to be written to:
+
+human-readable plaintext form:
+```shell
+kubectl explain pods --template /path/to/template.tmpl
+```
+
+Since the API surface for this sort of feature remains very unclear and will likely
+be very unstable, this sort of feature should be delayed until the internal
+templates have proven the API surface to be used. To do otherwise would risk
+breaking user's templates.
diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
index 27569805bb6..382f06e5225 100755
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
@@ -2,7 +2,7 @@ id: "3515"
 prnumber: ""
 name: kubectl-explain-openapiv3
 title: Kubectl Explain OpenAPIv3
-kep-number: draft
+kep-number: 3515
 authors: ['@alexzielenski']
 owning-sig: sig-cli
 participating-sigs: [sig-api-machinery, sig-cli]
@@ -11,7 +11,7 @@ approvers: ['@TBD']
 prr-approvers: []
 creation-date: "2022-09-14"
 last-updated: v1.19
-status: provisional
+status: implementable
 stage: alpha
 latest-milestone: ""
 milestone:

From 56fe61e1d0eea415a6f36fbded9583befff57400 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Wed, 28 Sep 2022 13:44:40 -0700
Subject: [PATCH 04/23] fix typos

---
 .../3515-kubectl-explain-openapiv3/README.md   | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index dcd4d28dab8..5920349a024 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -80,25 +80,24 @@ tags, and then generate with `hack/update-toc.sh`.
 - [Release Signoff Checklist](#release-signoff-checklist)
 - [Summary](#summary)
 - [Motivation](#motivation)
-  - [Richer OpenAPI V3 Data](#richer-openapi-v3-data)
+  - [OpenAPI v3 is a richer API description than OpenAPI v2](#openapi-v3-is-a-richer-api-description-than-openapi-v2)
   - [CRD schemas expressed as OpenAPI v2 are lossy](#crd-schemas-expressed-as-openapi-v2-are-lossy)
   - [Goals](#goals)
   - [Non-Goals](#non-goals)
 - [Proposal](#proposal)
   - [Basic Usage](#basic-usage)
   - [Built-in Template Options](#built-in-template-options)
-  - [Custom Templates](#custom-templates)
-    - [Example template](#example-template)
+    - [Plaintext](#plaintext)
+    - [Raw](#raw)
+    - [HTML](#html)
+    - [Markdown](#markdown)
   - [Risks and Mitigations](#risks-and-mitigations)
     - [OpenAPI V3 Not Available](#openapi-v3-not-available)
       - [Risk](#risk)
       - [Mitigation](#mitigation)
-    - [Template API Stability](#template-api-stability)
+    - [OpenAPI serialization time](#openapi-serialization-time)
       - [Risk](#risk-1)
       - [Mitigation](#mitigation-1)
-    - [OpenAPI serialization time](#openapi-serialization-time)
-      - [Risk](#risk-2)
-      - [Mitigation](#mitigation-2)
 - [Design Details](#design-details)
     - [Current High-level Approach](#current-high-level-approach)
     - [Proposed High-level Approach](#proposed-high-level-approach)
@@ -125,6 +124,7 @@ tags, and then generate with `hack/update-toc.sh`.
 - [Drawbacks](#drawbacks)
 - [Alternatives](#alternatives)
   - [Implement proto.Models for OpenAPI V3 data](#implement-protomodels-for-openapi-v3-data)
+  - [Custom User Templates](#custom-user-templates)
 <!-- /toc -->
 
 ## Release Signoff Checklist
@@ -783,7 +783,7 @@ Yes, up feature replaces a single GET of `/openapi/v2` which returns a large (me
 openapi document for all types with a more targeted call to `/openapi/v3/<group>/<version>`
 
 The `/openapi/v3/<group>/<version>` endpoint implements E-Tag caching so that if the document has
-not changed the server incurs a cheap, almost neglible cost to serving the request.
+not changed the server incurs a cheap, almost negligible cost to serving the request.
 
 The document returned by calls to `/openapi/v3/...` is expected to be far smaller
 than the megabytes-scale openapi v2 document, since it only includes information
@@ -882,7 +882,7 @@ Why should this KEP _not_ be implemented?
 The current hard-coded printer is capable of printing any objects in `proto.Models` form.
 
 [We already have a way to express OpenAPI v3 data as `proto.Models`, so this can be
-seen as a path of least resistence for plugging OpenAPI v3 into `kubectl explain`.
+seen as a path of least resistance for plugging OpenAPI v3 into `kubectl explain`.
 
 This approach is undesirable for a few different reasons:
 

From 9fcaad4753840c64012b367e5f35cad3f7bd6937 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Thu, 29 Sep 2022 10:22:22 -0700
Subject: [PATCH 05/23] fixx out kep yaml

---
 .../3515-kubectl-explain-openapiv3/kep.yaml   | 20 +++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
index 382f06e5225..1b1b5faca62 100755
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
@@ -1,23 +1,27 @@
 id: "3515"
-prnumber: ""
+prnumber: "3516"
 name: kubectl-explain-openapiv3
 title: Kubectl Explain OpenAPIv3
 kep-number: 3515
 authors: ['@alexzielenski']
 owning-sig: sig-cli
 participating-sigs: [sig-api-machinery, sig-cli]
-reviewers: []
-approvers: ['@TBD']
-prr-approvers: []
+reviewers:
+  - "@KnVerey"
+  - "@seans3"
+  - "@apelisse"
+approvers:
+  - "@KnVerey"
+  - "@seans3"
 creation-date: "2022-09-14"
 last-updated: v1.19
 status: implementable
 stage: alpha
-latest-milestone: ""
+latest-milestone: "1.26"
 milestone:
-    alpha: ""
-    beta: ""
-    stable: ""
+    alpha: "1.26"
+    beta: "1.28"
+    stable: "1.29"
 feature-gates: []
 disable-supported: false
 metrics: []

From 677ccacc7ea94bd7b9d7ffc6d9b312d234615662 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Thu, 29 Sep 2022 10:31:08 -0700
Subject: [PATCH 06/23] remove prnumber

what is this used for
---
 keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
index 1b1b5faca62..e9103663d7c 100755
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
@@ -1,5 +1,4 @@
 id: "3515"
-prnumber: "3516"
 name: kubectl-explain-openapiv3
 title: Kubectl Explain OpenAPIv3
 kep-number: 3515

From 6fa9647c6e0b6676b241f480285e61a4a4c8512f Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Thu, 29 Sep 2022 16:06:44 -0700
Subject: [PATCH 07/23] fill in testing plan a bit more

---
 .../3515-kubectl-explain-openapiv3/README.md  | 22 ++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 5920349a024..1071f107d96 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -385,7 +385,7 @@ when drafting this test plan.
 [testing-guidelines]: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
 -->
 
-[ ] I/we understand the owners of the involved components may require updates to
+[x] I/we understand the owners of the involved components may require updates to
 existing tests to make this code solid enough prior to committing the changes necessary
 to implement this enhancement.
 
@@ -417,7 +417,7 @@ This can inform certain test coverage improvements that we want to do before
 extending the production code to implement this enhancement.
 -->
 
-- `<package>`: `<date>` - `<test coverage>`
+- `k8s.io/kubectl/pkg/explain`: `09/29/2022`-`75.6`
 
 ##### Integration tests
 
@@ -431,6 +431,13 @@ https://storage.googleapis.com/k8s-triage/index.html
 
 - <test>: <link to test coverage>
 
+Tests should include
+
+- Expected Output tests
+- Show correct OpenAPI v3 endpoints are hit
+- Tests that show default/nullability information is being included in plaintext output
+- Tests that update the backing openapi in between calls to explain
+
 ##### e2e tests
 
 <!--
@@ -443,6 +450,11 @@ https://storage.googleapis.com/k8s-triage/index.html
 We expect no non-infra related flakes in the last month as a GA graduation criteria.
 -->
 
+Existing e2e tests should be adapted for the new system.
+E2E test that shows every definition in OpenAPI document can be retrieved via explain
+
+
+
 - <test>: <link to test coverage>
 
 ### Graduation Criteria
@@ -453,10 +465,14 @@ Defined using feature gate
 
 - Feature implemented behind a command line flag `--experimental-openapiv3`
 - Existing explain tests are working or adapted for new implementation
+- Plaintext output roughly matches explain output
+- Raw output implemented
 
 #### Beta
 
-- kube-OpenAPI v3 JSON deserialization is optimized to take less than 150ms on 
+- md output implemented
+- basic html output
+- kube-OpenAPI v3 JSON deserialization is optimized to take less than 150ms on
   most machines
 
 #### GA

From e8382b4ebe70d92370eba1a7a859ad1953a4bde7 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Fri, 30 Sep 2022 10:04:18 -0700
Subject: [PATCH 08/23] rename raw to openapiv3

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 1071f107d96..d269a0b0941 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -272,10 +272,10 @@ kubectl explain pods --output plaintext
 The plaintext output format is the default and should be crafted to be as close
 as the existing `explain` output in use before this KEP.
 
-#### Raw
+#### OpenAPIV3 (raw json)
 
 ```shell
-kubectl explain pods --output raw
+kubectl explain pods --output openapiv3
 ```
 
 To get raw OpenAPI v3 data for a certain resource today involves:
@@ -466,7 +466,7 @@ Defined using feature gate
 - Feature implemented behind a command line flag `--experimental-openapiv3`
 - Existing explain tests are working or adapted for new implementation
 - Plaintext output roughly matches explain output
-- Raw output implemented
+- OpenAPIV3 (raw json) output implemented
 
 #### Beta
 
@@ -713,10 +713,10 @@ logs or events for this purpose.
 ###### How can someone using this feature know that it is working for their instance?
 
 ```shell
-kubectl explain pods --output raw
+kubectl explain pods --output openapiv3
 ```
 
-TBD exact output. User should see OpenAPI v3 JSON Schema for `pods` type printed to console.
+User should see OpenAPI v3 JSON Schema for `pods` type printed to console.
 
 ###### What are the reasonable SLOs (Service Level Objectives) for the enhancement?
 

From 9f744f3adbc5a512144149ea253a432d983df654 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Fri, 30 Sep 2022 11:33:32 -0700
Subject: [PATCH 09/23] update with feedback

---
 .../3515-kubectl-explain-openapiv3/README.md  | 32 +++++++++++++++----
 1 file changed, 25 insertions(+), 7 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index d269a0b0941..3329f0f7fe5 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -321,11 +321,26 @@ OpenAPI v3 data is not available in the current cluster.
 
 ##### Mitigation
 
+# If the user does not provide an --output argument
+
+This is a "compatibility mode" where users will see OpenAPI v3 plaintext template
+if the data is available, or fallback to old `kubectl explain` behavior for openapi v2.
+
 If the initial request for OpenAPI V3 fails, the old implementation using
 OpenAPI v2 should be used instead. There is another risk that this fallback process
 takes too long for interactive speed, so timeout should be carefully considered
-to prevent user from waiting too long to see their results. 
+to prevent user from waiting too long to see their results.
+
+# If the user does provider an --output argument
 
+If a user specifies an `--output` argument and the server 404's attempting to
+fetch the correct openapi version for the template, a new error message should
+be thrown to the effect of server missing openapi data for version: %v.%v.%v`.
+
+Internal templates should strive to support all OpenAPI versions supported by
+versoins of kubernetes within their skew.
+
+Other network errors should be handled using normal kubectl error handling.
 
 #### OpenAPI serialization time
 ##### Risk
@@ -353,8 +368,10 @@ TODO: fill with links to referred implementation
 6. kubectl converts `gnostic_v2.Document` into `proto.Models`
 7. kubectl searches the document's `Definitions` for a schema with the
 extension `x-kubernetes-group-version-kind` matching the interested GVK
-8. kubectl renders the definition using its hardcoded printer
-9. If `--recursive` was used, repeat step 8 for the transitive closure of
+8. If a field path was used, kubectl traveres the definition's fields to the subschema
+specified by the user's path.
+9. kubectl renders the definition using its hardcoded printer
+10. If `--recursive` was used, repeat step 9 for the transitive closure of
   object-typed fields of the top-level object. Concat the results together.
 
 #### Proposed High-level Approach
@@ -363,10 +380,11 @@ extension `x-kubernetes-group-version-kind` matching the interested GVK
 2. kubectl resolves 'pods' to GVR core v1 pods using cluster discovery information
 3. kubectl fetches `/openapi/v3/<group>/<version>`
 4. kubectl parses the result as kube-openapi spec3
-5. kubectl locates the return type of the `/apis/<group>/<version>/<resource>` OpenAPI Path
-6. kubectl renders the type using its built-in template for human-readable plaintext
-7. If `--recursive` was used, repeat step 6 for the transitive closure of
-  object-typed fields of the top-level object. Concat the results together.
+5. kubectl locates the schema of the return type for the Path `/apis/<group>/<version>/<resource>` in kube-openapi
+6. If a field path was used, kubectl traveres the definition's fields to the subschema
+specified by the user's path.
+8. kubectl renders the type using its built-in template for human-readable plaintext
+9. If `--recursive` was used, repeat step 8 for the transitive closure of object-typed fields of the top-level object. Concat the results together.
 
 ### Template rendering
 

From 0316076bf98b7d5ae10977c193a9a8af532794da Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Fri, 30 Sep 2022 11:39:21 -0700
Subject: [PATCH 10/23] fix typo

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 3329f0f7fe5..bf8e2ec1e7f 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -368,7 +368,7 @@ TODO: fill with links to referred implementation
 6. kubectl converts `gnostic_v2.Document` into `proto.Models`
 7. kubectl searches the document's `Definitions` for a schema with the
 extension `x-kubernetes-group-version-kind` matching the interested GVK
-8. If a field path was used, kubectl traveres the definition's fields to the subschema
+8. If a field path was used, kubectl traverses the definition's fields to the subschema
 specified by the user's path.
 9. kubectl renders the definition using its hardcoded printer
 10. If `--recursive` was used, repeat step 9 for the transitive closure of

From 15fb8cdf592c90cafb2aed66bf7008daf62b8f9e Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Fri, 30 Sep 2022 12:00:15 -0700
Subject: [PATCH 11/23] fill in version skew strategy

---
 .../3515-kubectl-explain-openapiv3/README.md  | 49 ++++++++++++++-----
 1 file changed, 36 insertions(+), 13 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index bf8e2ec1e7f..0b5cfa7d7a2 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -243,7 +243,7 @@ wiping nullable fields.
 The following user experience should be possible with `kubectl explain`
 
 ```shell
-kubectl explain pods
+kubectl explain pods.spec
 ```
 
 Output should be familiar to users of today's `kubectl explain`, except new
@@ -321,7 +321,7 @@ OpenAPI v3 data is not available in the current cluster.
 
 ##### Mitigation
 
-# If the user does not provide an --output argument
+###### If the user does not provide an --output argument
 
 This is a "compatibility mode" where users will see OpenAPI v3 plaintext template
 if the data is available, or fallback to old `kubectl explain` behavior for openapi v2.
@@ -331,14 +331,14 @@ OpenAPI v2 should be used instead. There is another risk that this fallback proc
 takes too long for interactive speed, so timeout should be carefully considered
 to prevent user from waiting too long to see their results.
 
-# If the user does provider an --output argument
+###### If the user does provider an --output argument
 
 If a user specifies an `--output` argument and the server 404's attempting to
 fetch the correct openapi version for the template, a new error message should
 be thrown to the effect of server missing openapi data for version: %v.%v.%v`.
 
 Internal templates should strive to support all OpenAPI versions supported by
-versoins of kubernetes within their skew.
+versions of kubernetes within their skew.
 
 Other network errors should be handled using normal kubectl error handling.
 
@@ -472,7 +472,6 @@ Existing e2e tests should be adapted for the new system.
 E2E test that shows every definition in OpenAPI document can be retrieved via explain
 
 
-
 - <test>: <link to test coverage>
 
 ### Graduation Criteria
@@ -482,15 +481,24 @@ Defined using feature gate
 #### Alpha
 
 - Feature implemented behind a command line flag `--experimental-openapiv3`
+- `--output` flag added
 - Existing explain tests are working or adapted for new implementation
 - Plaintext output roughly matches explain output
 - OpenAPIV3 (raw json) output implemented
+- HTML and MD outputs are not target for alpha
 
 #### Beta
 
-- md output implemented
-- basic html output
-- kube-OpenAPI v3 JSON deserialization is optimized to take less than 150ms on
+- md output implemented (or dropped from design due to continued debate)
+  - Table of contents all GVKs grouped by Group then Version.
+  - Section for each individual GVK
+  - All types hyperlink to specific section
+- basic html output  (or dropped from design due to continued debate)
+  - Table of contents all GVKs grouped by Group then Version.
+  - Page for each individual GVK.
+  - All types hyperlink to their specific page
+  - Searchable by name, description, field name.
+- kube-openAPI v3 JSON deserialization is optimized to take less than 150ms on
   most machines
 
 #### GA
@@ -571,6 +579,8 @@ enhancement:
   cluster required to make on upgrade, in order to make use of the enhancement?
 -->
 
+N/A
+
 ### Version Skew Strategy
 
 <!--
@@ -586,6 +596,18 @@ enhancement:
   CRI or CNI may require updating that component before the kubelet.
 -->
 
+This feature only requires the target cluster has enabled The OpenAPIV3 feature.
+
+OpenAPIV3 is Beta as of Kubernetes 1.24. This feature should not be on-by-default
+until it is GA.
+
+Users of the `--output` flag who attempt to use it against a cluster for which
+OpenAPI v3 is not enabled will be shown an error informing them of missing openapi
+version upon 404.
+
+Built-in templates supported by kubectl should aim to support any OpenAPI
+version which is GA.
+
 ## Production Readiness Review Questionnaire
 
 <!--
@@ -645,17 +667,18 @@ Any change of default behavior may be surprising to users or break existing
 automations, so be extremely careful here.
 -->
 
-###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
+Enabling the feature changes the data source of `kubectl explain` to use openapiv3.
+The output optimally should be familiar to users, who may be delighted to see new
+information populated.
 
-Yes. The old hand-written printer for OpenAPI v2 data in use today can be used as
-a fallback.
+###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
 
 Until the feature is stable it will only be enabled when the `--experimental-openapiv3` flag is used.
+It has no persistent effect on data that is viewewd.
 
 ###### What happens if we reenable the feature if it was previously rolled back?
 
-There is no persistence to using the `--experimental-openapiv3` flag. It is only
-used for viewing data.
+There is no persistence to using the feature. It is only used for viewing data.
 
 ###### Are there any tests for feature enablement/disablement?
 

From f2d6ffd443176d8d79e14eb4f2d456bdb3345a9b Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Fri, 30 Sep 2022 12:03:44 -0700
Subject: [PATCH 12/23] update toc

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 0b5cfa7d7a2..fd3888ff8a3 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -88,7 +88,7 @@ tags, and then generate with `hack/update-toc.sh`.
   - [Basic Usage](#basic-usage)
   - [Built-in Template Options](#built-in-template-options)
     - [Plaintext](#plaintext)
-    - [Raw](#raw)
+    - [OpenAPIV3 (raw json)](#openapiv3-raw-json)
     - [HTML](#html)
     - [Markdown](#markdown)
   - [Risks and Mitigations](#risks-and-mitigations)

From 9422a7dde2c613a352d21ae6f7d6547e7df9f73b Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Fri, 30 Sep 2022 12:06:02 -0700
Subject: [PATCH 13/23] formatting fixes

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index fd3888ff8a3..d1dd9b85ed5 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -331,11 +331,11 @@ OpenAPI v2 should be used instead. There is another risk that this fallback proc
 takes too long for interactive speed, so timeout should be carefully considered
 to prevent user from waiting too long to see their results.
 
-###### If the user does provider an --output argument
+###### If the user does provide an --output argument
 
 If a user specifies an `--output` argument and the server 404's attempting to
 fetch the correct openapi version for the template, a new error message should
-be thrown to the effect of server missing openapi data for version: %v.%v.%v`.
+be thrown to the effect of: `server missing openapi data for version: %v.%v.%v`.
 
 Internal templates should strive to support all OpenAPI versions supported by
 versions of kubernetes within their skew.
@@ -381,7 +381,7 @@ specified by the user's path.
 3. kubectl fetches `/openapi/v3/<group>/<version>`
 4. kubectl parses the result as kube-openapi spec3
 5. kubectl locates the schema of the return type for the Path `/apis/<group>/<version>/<resource>` in kube-openapi
-6. If a field path was used, kubectl traveres the definition's fields to the subschema
+6. If a field path was used, kubectl traverses the definition's fields to the subschema
 specified by the user's path.
 8. kubectl renders the type using its built-in template for human-readable plaintext
 9. If `--recursive` was used, repeat step 8 for the transitive closure of object-typed fields of the top-level object. Concat the results together.
@@ -894,7 +894,8 @@ details). For now, we leave it here.
 
 ###### How does this feature react if the API server and/or etcd is unavailable?
 
-
+Using kubectl's normal error handling. There is no lasting effect to data or the
+user.
 
 ###### What are other known failure modes?
 
@@ -913,6 +914,8 @@ For each of them, fill in the following information by copying the below templat
 
 ###### What steps should be taken if SLOs are not being met to determine the problem?
 
+N/A
+
 ## Implementation History
 
 <!--

From b7497e26e382f53fb841bfc2edcb253a8229a77d Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 14:02:05 -0700
Subject: [PATCH 14/23] remove TODOs

---
 .../3515-kubectl-explain-openapiv3/README.md        | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index d1dd9b85ed5..48dcf6e9834 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -247,11 +247,7 @@ kubectl explain pods.spec
 ```
 
 Output should be familiar to users of today's `kubectl explain`, except new
-information is populated:
-
-```console
-<TODO>
-```
+information from the OpenAPI v3 spec is now populated.
 
 Note: Feature during development will be gated by an experimental flag. The commands
 shown here elide the experimental flag for clarity.
@@ -306,11 +302,6 @@ kubectl explain pods --output md
 When using the `md` template, a markdown document is printed to stdout, so it
 might be saved and used for a documentation website, for example.
 
-```console
-<TODO>
-```
-
-
 ### Risks and Mitigations
 
 #### OpenAPI V3 Not Available
@@ -358,8 +349,6 @@ should be blocked on the merging of this optimization.
 
 #### Current High-level Approach
 
-TODO: fill with links to referred implementation
-
 1. User types `kubectl explain pods`
 2. kubectl resolves 'pods' to GVR core v1 pods using cluster discovery information
 3. kubectl resolves GVR to its GVK using restmapper

From c22718bcbc51b15344b3d9415666d7a2de4e7cb3 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 15:12:45 -0700
Subject: [PATCH 15/23] address feedback

---
 .../3515-kubectl-explain-openapiv3/README.md      | 15 +++++++++++++--
 .../3515-kubectl-explain-openapiv3/kep.yaml       |  4 ++--
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 48dcf6e9834..b56b39aab50 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -328,11 +328,15 @@ If a user specifies an `--output` argument and the server 404's attempting to
 fetch the correct openapi version for the template, a new error message should
 be thrown to the effect of: `server missing openapi data for version: %v.%v.%v`.
 
-Internal templates should strive to support all OpenAPI versions supported by
-versions of kubernetes within their skew.
+Internal templates should strive to support the latest OpenAPI version enabled
+by default by versions of kubernetes within their skew. With that policy, templates
+will always render with the latest spec-version of the data, if it is available.
 
 Other network errors should be handled using normal kubectl error handling.
 
+During the experimental phase, specifying `--output` implies `--experimental-openapiv3`,
+so if OpenAPIv3 is unavailable, the fallback will not be tried. (fallback case is only in 'compatibility mode').
+
 #### OpenAPI serialization time
 ##### Risk
 
@@ -489,9 +493,16 @@ Defined using feature gate
   - Searchable by name, description, field name.
 - kube-openAPI v3 JSON deserialization is optimized to take less than 150ms on
   most machines
+- OpenAPI V3 is enabled by default on at least one version within kubectl's support window
+- Experimental flag is removed/made on by default (thus openapi v3 will always be tried first)
+- Old `kubectl explain` implementation for OpenAPI v2 remains as a fallback if v3 is unavailable
+(this policy should stand only until kubectl's version skew includes apiserver versions which enabled OpenAPI V3 by default). 
 
 #### GA
 
+- All kube-apiserver releases within version skew of kubectl should have OpenAPIV3 on by default
+- Old `kubectl explain` implementation is removed, as is support for OpenAPIV2-backed `kubectl explain`
+
 <!--
 **Note:** *Not required until targeted at a release.*
 
diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
index e9103663d7c..c6f3b5eba08 100755
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/kep.yaml
@@ -13,7 +13,7 @@ approvers:
   - "@KnVerey"
   - "@seans3"
 creation-date: "2022-09-14"
-last-updated: v1.19
+last-updated: v1.26
 status: implementable
 stage: alpha
 latest-milestone: "1.26"
@@ -22,5 +22,5 @@ milestone:
     beta: "1.28"
     stable: "1.29"
 feature-gates: []
-disable-supported: false
+disable-supported: true
 metrics: []

From 35e970ee62b476f680d389bf9bcb5c97d11390dd Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 15:48:26 -0700
Subject: [PATCH 16/23] address feedback

---
 .../3515-kubectl-explain-openapiv3/README.md  | 28 ++++++++++++++-----
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index b56b39aab50..8d2330846a8 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -225,6 +225,8 @@ wiping nullable fields.
     * maybe others
 5. (Optional?) Allow users to specify their own templates for use with kubectl
   explain (there may be interesting use cases for this)
+6. Improve discoverability of API Resources and endpoints, and provide a platform
+for richer information to be included in the future.
 
 ### Non-Goals
 
@@ -289,10 +291,16 @@ template solution in the future.
 kubectl explain pods --output html
 ```
 
-Similarly to [godoc](https://pkg.go.dev), the we suggest to provide a searchable,
+Similarly to [godoc](https://pkg.go.dev), we suggest to provide a searchable,
 navigable, generated webpage for the kubernetes types of whatever cluster kubectl
 is talking to.
 
+Only the fields selected in the command line (and their subfields' types, etc) 
+will be included in the resultant page.
+
+If user types `kubectl explain --output html` with no specific target, then all types
+in the cluster are included.
+
 #### Markdown
 
 ```shell
@@ -302,6 +310,12 @@ kubectl explain pods --output md
 When using the `md` template, a markdown document is printed to stdout, so it
 might be saved and used for a documentation website, for example.
 
+Similary to `html` output, only the fields selected in the command line 
+(and their subfields' types, etc) will be included in the resultant page.
+
+If user types `kubectl explain --output md` with no specific target, then all types
+in the cluster are included.
+
 ### Risks and Mitigations
 
 #### OpenAPI V3 Not Available
@@ -334,8 +348,6 @@ will always render with the latest spec-version of the data, if it is available.
 
 Other network errors should be handled using normal kubectl error handling.
 
-During the experimental phase, specifying `--output` implies `--experimental-openapiv3`,
-so if OpenAPIv3 is unavailable, the fallback will not be tried. (fallback case is only in 'compatibility mode').
 
 #### OpenAPI serialization time
 ##### Risk
@@ -473,8 +485,8 @@ Defined using feature gate
 
 #### Alpha
 
-- Feature implemented behind a command line flag `--experimental-openapiv3`
-- `--output` flag added
+- Feature implemented behind a command line flag `--experimental-output`
+- `--experimental-output` flag added
 - Existing explain tests are working or adapted for new implementation
 - Plaintext output roughly matches explain output
 - OpenAPIV3 (raw json) output implemented
@@ -500,6 +512,7 @@ Defined using feature gate
 
 #### GA
 
+- `--experimental-output` renamed to `--output`
 - All kube-apiserver releases within version skew of kubectl should have OpenAPIV3 on by default
 - Old `kubectl explain` implementation is removed, as is support for OpenAPIV2-backed `kubectl explain`
 
@@ -654,7 +667,8 @@ well as the [existing list] of feature gates.
   - Feature gate name:
   - Components depending on the feature gate:
 - [x] Other
-  - Describe the mechanism: --experimental-openapiv3 flag
+  - Describe the mechanism: --experimental-output flag usage
+    (to be renamed to --output when feature is no longer experimental)
   - Will enabling / disabling the feature require downtime of the control
     plane? No
   - Will enabling / disabling the feature require downtime or reprovisioning
@@ -673,7 +687,7 @@ information populated.
 
 ###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
 
-Until the feature is stable it will only be enabled when the `--experimental-openapiv3` flag is used.
+Until the feature is stable it will only be enabled when the `--experimental-output` flag is used.
 It has no persistent effect on data that is viewewd.
 
 ###### What happens if we reenable the feature if it was previously rolled back?

From 9af3bcece09c153ee98bd292b4bab47786a56843 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 15:52:50 -0700
Subject: [PATCH 17/23] clarify --output fallback

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 8d2330846a8..d33d13c7016 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -328,13 +328,12 @@ OpenAPI v3 data is not available in the current cluster.
 
 ###### If the user does not provide an --output argument
 
-This is a "compatibility mode" where users will see OpenAPI v3 plaintext template
-if the data is available, or fallback to old `kubectl explain` behavior for openapi v2.
+While this feature is not GA, this case should fallback to the old OpenAPI v2
+`kubectl explain` implementation if the server responds with `404` for OpenAPIV3 data.
 
-If the initial request for OpenAPI V3 fails, the old implementation using
-OpenAPI v2 should be used instead. There is another risk that this fallback process
-takes too long for interactive speed, so timeout should be carefully considered
-to prevent user from waiting too long to see their results.
+Once this feature is GA, then `OpenAPIV3` should be available everywhere
+so this is not a concern. If a user uses the feature against such a cluster without
+OpenAPIV3 after this KEP is GA, an error will be shown.
 
 ###### If the user does provide an --output argument
 

From ebb5e29ccb10294be4541761ec74fc2de22b2c7e Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 15:54:30 -0700
Subject: [PATCH 18/23] spelling

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index d33d13c7016..1cf5e92bbd0 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -310,7 +310,7 @@ kubectl explain pods --output md
 When using the `md` template, a markdown document is printed to stdout, so it
 might be saved and used for a documentation website, for example.
 
-Similary to `html` output, only the fields selected in the command line 
+Similarly to `html` output, only the fields selected in the command line 
 (and their subfields' types, etc) will be included in the resultant page.
 
 If user types `kubectl explain --output md` with no specific target, then all types

From 420ec131c367c7d143af260883c968f1af6af570 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 19:18:36 -0700
Subject: [PATCH 19/23] address feedback about rollout

---
 .../3515-kubectl-explain-openapiv3/README.md  | 46 +++++++++++--------
 1 file changed, 27 insertions(+), 19 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 1cf5e92bbd0..e5c17d950df 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -108,7 +108,8 @@ tags, and then generate with `hack/update-toc.sh`.
       - [Integration tests](#integration-tests)
       - [e2e tests](#e2e-tests)
   - [Graduation Criteria](#graduation-criteria)
-    - [Alpha](#alpha)
+    - [Alpha 1](#alpha-1)
+    - [Alpha 2](#alpha-2)
     - [Beta](#beta)
     - [GA](#ga)
   - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
@@ -287,6 +288,8 @@ template solution in the future.
 
 #### HTML
 
+> PROVISIONAL SECTION
+
 ```shell
 kubectl explain pods --output html
 ```
@@ -298,10 +301,11 @@ is talking to.
 Only the fields selected in the command line (and their subfields' types, etc) 
 will be included in the resultant page.
 
-If user types `kubectl explain --output html` with no specific target, then all types
-in the cluster are included.
+Possible idea: If user types `kubectl explain --output html` with no specific target, 
+then all types in the cluster are included.
 
 #### Markdown
+> PROVISIONAL SECTION
 
 ```shell
 kubectl explain pods --output md
@@ -313,8 +317,8 @@ might be saved and used for a documentation website, for example.
 Similarly to `html` output, only the fields selected in the command line 
 (and their subfields' types, etc) will be included in the resultant page.
 
-If user types `kubectl explain --output md` with no specific target, then all types
-in the cluster are included.
+Possible idea: If user types `kubectl explain --output md` with no specific target,
+then all types in the cluster are included.
 
 ### Risks and Mitigations
 
@@ -482,37 +486,39 @@ E2E test that shows every definition in OpenAPI document can be retrieved via ex
 
 Defined using feature gate
 
-#### Alpha
+#### Alpha 1
 
 - Feature implemented behind a command line flag `--experimental-output`
 - `--experimental-output` flag added
 - Existing explain tests are working or adapted for new implementation
 - Plaintext output roughly matches explain output
 - OpenAPIV3 (raw json) output implemented
-- HTML and MD outputs are not target for alpha
-
-#### Beta
 
-- md output implemented (or dropped from design due to continued debate)
+#### Alpha 2
+- `md` output implemented (or dropped from design due to continued debate)
   - Table of contents all GVKs grouped by Group then Version.
   - Section for each individual GVK
   - All types hyperlink to specific section
-- basic html output  (or dropped from design due to continued debate)
+-  basic `html` output  (or dropped from design due to continued debate)
   - Table of contents all GVKs grouped by Group then Version.
   - Page for each individual GVK.
   - All types hyperlink to their specific page
   - Searchable by name, description, field name.
+
+#### Beta
+
 - kube-openAPI v3 JSON deserialization is optimized to take less than 150ms on
   most machines
-- OpenAPI V3 is enabled by default on at least one version within kubectl's support window
-- Experimental flag is removed/made on by default (thus openapi v3 will always be tried first)
-- Old `kubectl explain` implementation for OpenAPI v2 remains as a fallback if v3 is unavailable
-(this policy should stand only until kubectl's version skew includes apiserver versions which enabled OpenAPI V3 by default). 
+- OpenAPI V3 is enabled by default on at least one version within kubectl's support window.
+As of Kubernetes 1.24 OpenAPIV3 entered beta and become enabled by default, therefore meeting this requirement.
+- `--experimental-output` flag is renamed to `--output`, and `--output plaintext` is on-by-default.
+- `--output plaintext-openapiv2` added as a name for the old `explain` implementation, so the feature may be positively disabled.
 
 #### GA
 
-- `--experimental-output` renamed to `--output`
-- All kube-apiserver releases within version skew of kubectl should have OpenAPIV3 on by default
+- OpenAPIV3 is GA and has been since at least the minimum supported apiserver version
+by kubectl.
+- All kube-apiserver releases within version skew of kubectl should have OpenAPIV3 on by default. This is true as of Kubernetes 1.24
 - Old `kubectl explain` implementation is removed, as is support for OpenAPIV2-backed `kubectl explain`
 
 <!--
@@ -617,8 +623,10 @@ Users of the `--output` flag who attempt to use it against a cluster for which
 OpenAPI v3 is not enabled will be shown an error informing them of missing openapi
 version upon 404.
 
-Built-in templates supported by kubectl should aim to support any OpenAPI
-version which is GA.
+Built-in templates supported by kubectl should aim to support at least one OpenAPI
+version which is GA for an apiserver version within the support window.
+`kubectl` will support trying to fetch each of these versions, so one is guaranteed
+to be able to render.
 
 ## Production Readiness Review Questionnaire
 

From 33f6e9f75a96a3ab65fa778e68f8a98ed5572840 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 19:29:58 -0700
Subject: [PATCH 20/23] add explicit not alpha 2 may be dropped

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index e5c17d950df..bb7f580bcdd 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -495,6 +495,10 @@ Defined using feature gate
 - OpenAPIV3 (raw json) output implemented
 
 #### Alpha 2
+
+If we decide to move ahead with the `md` and `html` outputs, an Alpha 2 may
+be required.
+
 - `md` output implemented (or dropped from design due to continued debate)
   - Table of contents all GVKs grouped by Group then Version.
   - Section for each individual GVK

From 1bd5b56f6b7dad11f0636ccb78fc49996c954619 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Mon, 3 Oct 2022 19:32:28 -0700
Subject: [PATCH 21/23] add note about openapi v3 data not available

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index bb7f580bcdd..f5b658a911b 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -332,12 +332,10 @@ OpenAPI v3 data is not available in the current cluster.
 
 ###### If the user does not provide an --output argument
 
-While this feature is not GA, this case should fallback to the old OpenAPI v2
-`kubectl explain` implementation if the server responds with `404` for OpenAPIV3 data.
+In alpha in particular, if `--output` is not specified, the old `explain` behavior
+using openapi v2 deta will be used.
 
-Once this feature is GA, then `OpenAPIV3` should be available everywhere
-so this is not a concern. If a user uses the feature against such a cluster without
-OpenAPIV3 after this KEP is GA, an error will be shown.
+After beta, `--output plaintext` will be assumed and behave as below.
 
 ###### If the user does provide an --output argument
 

From a58033b6dfa6df8963e40252e49219282ce8e319 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Tue, 4 Oct 2022 10:41:19 -0700
Subject: [PATCH 22/23] add GA criterion

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index f5b658a911b..06b00531d00 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -520,8 +520,9 @@ As of Kubernetes 1.24 OpenAPIV3 entered beta and become enabled by default, ther
 
 - OpenAPIV3 is GA and has been since at least the minimum supported apiserver version
 by kubectl.
-- All kube-apiserver releases within version skew of kubectl should have OpenAPIV3 on by default. This is true as of Kubernetes 1.24
+- All kube-apiserver releases within version skew of kubectl should have OpenAPIV3 on by default. This is true as of kubectl for Kubernetes 1.25
 - Old `kubectl explain` implementation is removed, as is support for OpenAPIV2-backed `kubectl explain`
+- `--output plaintext-openapiv2` has been deprecated for at least one release
 
 <!--
 **Note:** *Not required until targeted at a release.*

From f3fe05d0df03035c4a170680569b81012fbbc9b7 Mon Sep 17 00:00:00 2001
From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com>
Date: Wed, 5 Oct 2022 13:53:30 -0700
Subject: [PATCH 23/23] changed feature gate to use environment variable

---
 keps/sig-cli/3515-kubectl-explain-openapiv3/README.md | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
index 06b00531d00..63532363c3f 100644
--- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
+++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md
@@ -486,8 +486,7 @@ Defined using feature gate
 
 #### Alpha 1
 
-- Feature implemented behind a command line flag `--experimental-output`
-- `--experimental-output` flag added
+- Feature implemented behind a command line flag `--output` and environment variable
 - Existing explain tests are working or adapted for new implementation
 - Plaintext output roughly matches explain output
 - OpenAPIV3 (raw json) output implemented
@@ -513,7 +512,7 @@ be required.
   most machines
 - OpenAPI V3 is enabled by default on at least one version within kubectl's support window.
 As of Kubernetes 1.24 OpenAPIV3 entered beta and become enabled by default, therefore meeting this requirement.
-- `--experimental-output` flag is renamed to `--output`, and `--output plaintext` is on-by-default.
+- `--output plaintext` is on-by-default and environment variable is removed/on by default
 - `--output plaintext-openapiv2` added as a name for the old `explain` implementation, so the feature may be positively disabled.
 
 #### GA
@@ -677,7 +676,7 @@ well as the [existing list] of feature gates.
   - Feature gate name:
   - Components depending on the feature gate:
 - [x] Other
-  - Describe the mechanism: --experimental-output flag usage
+  - Describe the mechanism: Environment variable `ENABLE_EXPLAIN_OPENAPIV3` which toggles validity of `--output` flag
     (to be renamed to --output when feature is no longer experimental)
   - Will enabling / disabling the feature require downtime of the control
     plane? No
@@ -697,7 +696,7 @@ information populated.
 
 ###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
 
-Until the feature is stable it will only be enabled when the `--experimental-output` flag is used.
+Until the feature is stable it will only be enabled when the environment variable is used.
 It has no persistent effect on data that is viewewd.
 
 ###### What happens if we reenable the feature if it was previously rolled back?