Merge pull request #55168 from nikhita/customresources-subresources

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

apiextensions: add subresources for custom resources

Fixes #38113
Fixes #58778

**Related**:
- Proposal: kubernetes/community#913
- For custom resources to work with `kubectl scale`: kubernetes/kubernetes#58283

**Add types**:

- Add `CustomResourceSubResources` type to CRD.
    - Fix proto generation for `CustomResourceSubResourceStatus`: kubernetes/kubernetes#55970.
- Add feature gate for `CustomResourceSubResources`.
    - Update CRD strategy: if feature gate is disabled, this feature is dropped (i.e. set to `nil`).
- Add validation for `CustomResourceSubResources`:
    - `SpecReplicasPath` should not be empty and should be a valid json path under `.spec`. If there is no value under the given path in the CustomResource, the `/scale` subresource will return an error on GET.
    - `StatusReplicasPath` should not be empty and should be a valid json path under `.status`. If there is no value under the given path in the CustomResource, the status replica value in the /scale subresource will default to 0.
    - If present, `LabelSelectorPath` should be a valid json path. If there is no value under `LabelSelectorPath` in the CustomResource, the status label selector value in the `/scale` subresource will default to the empty string.
    - `ScaleGroupVersion` should be `autoscaling/v1`.
    - If `CustomResourceSubResources` is enabled, only `properties` is allowed under the root schema for CRD validation.

**Add status and scale subresources**:

- Use helper functions from `apimachinery/pkg/apis/meta/v1/unstructured/helpers.go`.
    - Improve error handling: kubernetes/kubernetes#56563, kubernetes/kubernetes#58215.
- Introduce Registry interface for storage.
- Update storage:
    - Introduce `CustomResourceStorage` which acts as storage for the custom resource and its status and scale subresources. Note: storage for status and scale is only enabled when the feature gate is enabled _and_ the respective fields are enabled in the CRD.
    - Introduce `StatusREST` and its `New()`, `Get()` and `Update()` methods.
    - Introduce `ScaleREST` and its `New()`, `Get()` and `Update()` methods.
        - Get and Update use the json paths from the CRD and use it to return an `autoscaling/v1.Scale` object.
- Update strategy:
    - In `PrepareForCreate`,
         - Clear `.status`.
         - Set `.metadata.generation` = 1
    - In `PrepareForUpdate`,
         - Do not update `.status`.
             - If both the old and new objects have `.status` and it is changed, set it back to its old value.
             - If the old object has a `.status` but the new object doesn't, set it to the old value.
             - If old object did not have a `.status` but the new object does, delete it.
         - Increment generation if spec changes i.e. in the following cases:
             - If both the old and new objects had `.spec` and it changed.
             - If the old object did not have `.spec` but the new object does.
             - If the old object had a `.spec` but the new object doesn't.
     - In `Validate` and `ValidateUpdate`,
        - ensure that values at `specReplicasPath` and `statusReplicasPath` are >=0 and < maxInt32.
        - make sure there are no errors in getting the value at all the paths.
    - Introduce `statusStrategy` with its methods.
        - In `PrepareForUpdate`:
            - Do not update `.spec`.
                - If both the old and new objects have `.spec` and it is changed, set it back to its old value.
                - If the old object has a `.spec` but the new object doesn't, set it to the old value.
                - If old object did not have a `.spec` but the new object does, delete it.
             - Do not update `.metadata`.
        - In `ValidateStatusUpdate`:
            - For CRD validation, validate only under `.status`.
            - Validate value at `statusReplicasPath` as above. If `labelSelectorPath` is a path under `.status`, then validate it as well.
- Plug into the custom resource handler:
    - Store all three storage - customResource, status and scale in `crdInfo`.
    - Use the storage as per the subresource in the request.
    - Use the validator as per the subresource (for status, only use the schema for `status`, if present).
    - Serve the endpoint as per the subresource - see `serveResource`, `serveStatus` and `serveScale`.
- Update discovery by adding the `/status` and `/scale` resources, if enabled.

**Add tests**:

- Add unit tests in `etcd_test.go`.
- Add integration tests.
    - In `subresources_test.go`, use the [polymporphic scale client](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/client-go/scale) to get and update `Scale`.
    -  Add a test to check everything works fine with yaml in `yaml_test.go`.

**Release note**:

```release-note
`/status` and `/scale` subresources are added for custom resources.
```

Kubernetes-commit: 6e856480c05424b5cd2cfcbec692a801b856ccb2

dynamic/client.go

@@ -280,6 +280,7 @@ func (rc *ResourceClient) Watch(opts metav1.ListOptions) (watch.Interface, error
 		Watch()
 }
 
+// Patch applies the patch and returns the patched resource.
 func (rc *ResourceClient) Patch(name string, pt types.PatchType, data []byte) (*unstructured.Unstructured, error) {
 	result := new(unstructured.Unstructured)
 	resourceName, subresourceName := rc.parseResourceSubresourceName()

scale/client_test.go

@@ -99,7 +99,7 @@ func fakeScaleClient(t *testing.T) (ScalesGetter, []schema.GroupResource) {
 
 	restMapperRes, err := discovery.GetAPIGroupResources(fakeDiscoveryClient)
 	if err != nil {
-		t.Fatalf("unexpected error while constructing resource list from fake discovery client: %v")
+		t.Fatalf("unexpected error while constructing resource list from fake discovery client: %v", err)
 	}
 	restMapper := discovery.NewRESTMapper(restMapperRes, apimeta.InterfacesForUnstructured)