diff --git a/keps/prod-readiness/sig-cli/3515.yaml b/keps/prod-readiness/sig-cli/3515.yaml index b2bb14812b0..189166feb8f 100644 --- a/keps/prod-readiness/sig-cli/3515.yaml +++ b/keps/prod-readiness/sig-cli/3515.yaml @@ -1,3 +1,5 @@ kep-number: 3515 alpha: approver: "@johnbelamaric" +beta: + approver: "@johnbelamaric" diff --git a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md index 63532363c3f..3e39fbe08f7 100644 --- a/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md +++ b/keps/sig-cli/3515-kubectl-explain-openapiv3/README.md @@ -89,15 +89,10 @@ tags, and then generate with `hack/update-toc.sh`. - [Built-in Template Options](#built-in-template-options) - [Plaintext](#plaintext) - [OpenAPIV3 (raw json)](#openapiv3-raw-json) - - [HTML](#html) - - [Markdown](#markdown) - [Risks and Mitigations](#risks-and-mitigations) - [OpenAPI V3 Not Available](#openapi-v3-not-available) - [Risk](#risk) - [Mitigation](#mitigation) - - [OpenAPI serialization time](#openapi-serialization-time) - - [Risk](#risk-1) - - [Mitigation](#mitigation-1) - [Design Details](#design-details) - [Current High-level Approach](#current-high-level-approach) - [Proposed High-level Approach](#proposed-high-level-approach) @@ -109,7 +104,6 @@ tags, and then generate with `hack/update-toc.sh`. - [e2e tests](#e2e-tests) - [Graduation Criteria](#graduation-criteria) - [Alpha 1](#alpha-1) - - [Alpha 2](#alpha-2) - [Beta](#beta) - [GA](#ga) - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) @@ -126,6 +120,10 @@ tags, and then generate with `hack/update-toc.sh`. - [Alternatives](#alternatives) - [Implement proto.Models for OpenAPI V3 data](#implement-protomodels-for-openapi-v3-data) - [Custom User Templates](#custom-user-templates) +- [Future Work](#future-work) + - [Other template outputs](#other-template-outputs) + - [HTML](#html) + - [Markdown](#markdown) ## Release Signoff Checklist @@ -222,7 +220,6 @@ wiping nullable fields. 3. Fallback to old `explain` implementation if cluster does not expose OpenAPI v3 data. 4. Provide multiple new output formats for kubectl explain: * human-readable plaintext - * markdown * maybe others 5. (Optional?) Allow users to specify their own templates for use with kubectl explain (there may be interesting use cases for this) @@ -286,40 +283,6 @@ This command is useful not only for its convenience, but also other visualizatio may be built upon the raw output if we opt not to support a first-class custom template solution in the future. -#### HTML - -> PROVISIONAL SECTION - -```shell -kubectl explain pods --output html -``` - -Similarly to [godoc](https://pkg.go.dev), we suggest to provide a searchable, -navigable, generated webpage for the kubernetes types of whatever cluster kubectl -is talking to. - -Only the fields selected in the command line (and their subfields' types, etc) -will be included in the resultant page. - -Possible idea: If user types `kubectl explain --output html` with no specific target, -then all types in the cluster are included. - -#### Markdown -> PROVISIONAL SECTION - -```shell -kubectl explain pods --output md -``` - -When using the `md` template, a markdown document is printed to stdout, so it -might be saved and used for a documentation website, for example. - -Similarly to `html` output, only the fields selected in the command line -(and their subfields' types, etc) will be included in the resultant page. - -Possible idea: If user types `kubectl explain --output md` with no specific target, -then all types in the cluster are included. - ### Risks and Mitigations #### OpenAPI V3 Not Available @@ -333,9 +296,13 @@ OpenAPI v3 data is not available in the current cluster. ###### If the user does not provide an --output argument In alpha in particular, if `--output` is not specified, the old `explain` behavior -using openapi v2 deta will be used. +using openapi v2 data will be used. + +In beta, kubectl will test if server publishes `/openapi/v3`. If it does, it will +proceed with the new renderer. If there is no endpoint published, kubectl will fall +back to the old v2 implemtation. -After beta, `--output plaintext` will be assumed and behave as below. +After GA, `--output plaintext` will be assumed and behave as below. ###### If the user does provide an --output argument @@ -350,18 +317,6 @@ will always render with the latest spec-version of the data, if it is available. Other network errors should be handled using normal kubectl error handling. -#### OpenAPI serialization time -##### Risk - -Today there is no interactive-speed way to deserialize protobuf or JSON openapi -v3 data into the kube-openapi format. - -##### Mitigation - -There has been recent progress in this area. To unmarshal kube-OpenAPI v3 is now able -to be done in a performant enough way to do it in the CLI. This KEP's beta release -should be blocked on the merging of this optimization. - ## Design Details #### Current High-level Approach @@ -384,13 +339,15 @@ specified by the user's path. 1. User types `kubectl explain pods` 2. kubectl resolves 'pods' to GVR core v1 pods using cluster discovery information -3. kubectl fetches `/openapi/v3//` -4. kubectl parses the result as kube-openapi spec3 -5. kubectl locates the schema of the return type for the Path `/apis///` in kube-openapi -6. If a field path was used, kubectl traverses the definition's fields to the subschema +3. kubectl attempts to fetch `/openapi/v3`, which indexes where to find specs for each GV +4. If failure and fallback to v2 is allowed, falls back to Step #3 of the "Current High-level Approach". +5. Otherwise, kubectl fetches OpenAPIV3 path for GV: `/openapi/v3//` +6. kubectl parses the result as map[string]any +7. kubectl locates the schema of the return type for the Path `/apis///` +8. If a field path was used, kubectl traverses the definition's fields to the subschema specified by the user's path. -8. kubectl renders the type using its built-in template for human-readable plaintext -9. If `--recursive` was used, repeat step 8 for the transitive closure of object-typed fields of the top-level object. Concat the results together. +9. kubectl renders the type using its built-in template for human-readable plaintext +10. If `--recursive` was used, repeat step 9 for the transitive closure of object-typed fields of the top-level object. Concat the results together. ### Template rendering @@ -491,27 +448,13 @@ Defined using feature gate - Plaintext output roughly matches explain output - OpenAPIV3 (raw json) output implemented -#### Alpha 2 - -If we decide to move ahead with the `md` and `html` outputs, an Alpha 2 may -be required. - -- `md` output implemented (or dropped from design due to continued debate) - - Table of contents all GVKs grouped by Group then Version. - - Section for each individual GVK - - All types hyperlink to specific section -- basic `html` output (or dropped from design due to continued debate) - - Table of contents all GVKs grouped by Group then Version. - - Page for each individual GVK. - - All types hyperlink to their specific page - - Searchable by name, description, field name. - #### Beta -- kube-openAPI v3 JSON deserialization is optimized to take less than 150ms on - most machines - OpenAPI V3 is enabled by default on at least one version within kubectl's support window. As of Kubernetes 1.24 OpenAPIV3 entered beta and become enabled by default, therefore meeting this requirement. +In Kubectl for release 1.25, all k8s versions within support window will be able to have OpenAPIV3 enabled. +However, the fallback is kept around since it may not always be enabled. + - `--output plaintext` is on-by-default and environment variable is removed/on by default - `--output plaintext-openapiv2` added as a name for the old `explain` implementation, so the feature may be positively disabled. @@ -519,9 +462,8 @@ As of Kubernetes 1.24 OpenAPIV3 entered beta and become enabled by default, ther - OpenAPIV3 is GA and has been since at least the minimum supported apiserver version by kubectl. -- All kube-apiserver releases within version skew of kubectl should have OpenAPIV3 on by default. This is true as of kubectl for Kubernetes 1.25 +- OpenAPIV3 should be stable for all k8s versions within skew. - Old `kubectl explain` implementation is removed, as is support for OpenAPIV2-backed `kubectl explain` -- `--output plaintext-openapiv2` has been deprecated for at least one release + +Plan to add more tests for enablement/disablement for beta. PR started here with +tests that toggle feature on and off and show feature works in both cases. + +https://github.com/kubernetes/kubernetes/blob/a62e52cf2ede71a7219c04569ee09f0410f709f0/staging/src/k8s.io/kubectl/pkg/cmd/explain/explain_test.go#L157-L219 + + ### Rollout, Upgrade and Rollback Planning +No, this is a user-interactive CLI feature. If users don't like it they can +use the old functionality by providing arguments `--output plaintext-openapiv2` ###### What specific metrics should inform a rollback? @@ -745,11 +701,8 @@ that might indicate a serious problem? ###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? - +This feature has no state in the cluster. Using `explain` on a cluster cannot +affect other users. ###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? @@ -757,6 +710,8 @@ are missing a bunch of machinery and tooling and can't do that now. Even if applying deprecation policies, they may still surprise some users. --> +No. + ### Monitoring Requirements +There is no direct metrics of `explain` users, but operators can indirectly +gauge usership by watching openapi v3 metrics. + ###### How can someone using this feature know that it is working for their instance? ```shell -kubectl explain pods --output openapiv3 +kubectl explain pods --output plaintext ``` User should see OpenAPI v3 JSON Schema for `pods` type printed to console. ###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? +N/A + +N/A ### Dependencies +None ###### Does this feature depend on any specific services running in the cluster? @@ -843,7 +805,7 @@ and creating new ones, as well as about cluster-level services (e.g. DNS): --> To reap the benefits of this feature, OpenAPI v3 is required, however OpenAPI v2 -data can be used as a fallback. +data can be used as a fallback. OpenAPI V3 is GA as of Kubernetes 1.27. ### Scalability @@ -867,7 +829,8 @@ not changed the server incurs a cheap, almost negligible cost to serving the req The document returned by calls to `/openapi/v3/...` is expected to be far smaller than the megabytes-scale openapi v2 document, since it only includes information -for a single group-version. +for a single group-version. Additionally, this new mechanism is far more cache-friendly +so the expectation is that far less data will need to be transferred. +No, it is client-side only and only uses a single standard HTTP connection. + ### Troubleshooting