Proposal: enhance the caching ability of Yurthub #231
Conversation
|
|
||
| - Delete [resourceToKindMap](https://github.com/openyurtio/openyurt/blob/4d7463a40801c29d09c4f7d10ba46b73cb019915/pkg/yurthub/cachemanager/cache_manager.go#L46) and [resourceToListKindMap](https://github.com/openyurtio/openyurt/blob/4d7463a40801c29d09c4f7d10ba46b73cb019915/pkg/yurthub/cachemanager/cache_manager.go#L62) so that we can remove the restriction that yurthub can only cache certain resources; | ||
| - Decode Custom Resources that are not registered in the scheme into `Unstructured` structures; | ||
| - The Custom Resource is distinguished from the core resource based on `request.URL`. The core resource uses the existing serializer and the Custom Resource uses the `unstructuredNegotiatedSerializer`. |
There was a problem hiding this comment.
How about use unstructuredNegotiatedSerializer for all of resources?
There was a problem hiding this comment.
Unstructured objects still have functioning TypeMeta features-- kind, version, etc, but for private properties that distinguish resources, such as specs, registered structures are required to parse them out in detail. If use unstructuredNegotiatedSerializer to decode all resource, the Yurthub will lose the flexibility to operate core resources.
Or if you mean to use only one NegotiatedSerializer to handle all the resources, our current idea is to later encapsulate the two kind of serializers into one and decide internally which one to use.
There was a problem hiding this comment.
Unstructured objects still have functioning TypeMeta features-- kind, version, etc, but for private properties that distinguish resources, such as specs, registered structures are required to parse them out in detail. If use unstructuredNegotiatedSerializer to decode all resource, the Yurthub will lose the flexibility to operate core resources.
Or if you mean to use only one NegotiatedSerializer to handle all the resources, our current idea is to later encapsulate the two kind of serializers into one and decide internally which one to use.
For the sake of flexibility to operate resources, we should use unstructuredNegotiatedSerializer uniformly, and then use NegotiatedSerializer only when resources need to be considered for detail.
|
|
||
| Currently Yurthub can only encode and decode resources under these paths: | ||
|
|
||
| ```go |
There was a problem hiding this comment.
Would you give the reference file path for following code?
There was a problem hiding this comment.
The reference is https://github.com/kubernetes/client-go/blob/master/kubernetes/scheme/register.go . I will add this reference in the proposal.
| Remove the restriction that only certain resources can be cached by removing [resourceToKindMap](https://github.com/openyurtio/openyurt/blob/4d7463a40801c29d09c4f7d10ba46b73cb019915/pkg/yurthub/cachemanager/cache_manager.go#L46) and [resourceToListKindMap](https://github.com/openyurtio/openyurt/blob/4d7463a40801c29d09c4f7d10ba46b73cb019915/pkg/yurthub/cachemanager/cache_manager.go#L62). Removing the above restriction requires solving the problem which is preserving the original GVK information while decoding the corresponding resources; | ||
|
|
||
| The reason why object misses the field of apiverison and kind is YurtHub using [serializer.WithoutConversionCodecFactory](https://github.com/alibaba/openyurt/blob/7095962687e2666a27d14e436b96ef893b228beb/pkg/yurthub/kubernetes/serializer/serializer.go#L48) as NegotiatedSerializer, which produces [WithoutVersionDecoder ](https://github.com/kubernetes/kubernetes/blob/994b5c6cc20bda188ac9bc33217f2dbc7ecf45bb/staging/src/k8s.io/apimachinery/pkg/runtime/helper.go#L256)and remove the apiversion/kind of object when serialized from http.response. The following code is the decoder that is invoked when the resource is decoded: | ||
|
|
There was a problem hiding this comment.
When should we use WithoutVersionDecoder? would you give some examples to describe how to use these two Decoder?
There was a problem hiding this comment.
Currently, YurtHub use WithoutConversionCodecFactory as NegotiatedSerializer. Because the WithoutConversionCodecFactory is a CodecFactory that will explicitly ignore requests to perform conversion, the GVK information will be ignored when this WithoutConversionCodecFactory use the decoder (which is this Decoder).
So we're currently adding the new CodecFactory wrapper which is WithVersionCodecFactory to replace the old WithoutConversionCodecFactory as negotiatedSerializer, and rewriting the original Decode function to keep the GVK information.
There was a problem hiding this comment.
Currently, YurtHub use WithoutConversionCodecFactory as NegotiatedSerializer. Because the WithoutConversionCodecFactory is a CodecFactory that will explicitly ignore requests to perform conversion, the GVK information will be ignored when this WithoutConversionCodecFactory use the decoder (which is this Decoder).
So we're currently adding the new CodecFactory wrapper which is WithVersionCodecFactory to replace the old WithoutConversionCodecFactory as negotiatedSerializer, and rewriting the original Decode function to keep the GVK information.
And when we can use WithoutVersionDecoder ?
qclc
force-pushed
the
Addproposal
branch
2 times, most recently
from
f51fb6b
to
cdf2cfd
Compare
|
FYI, CRD does not support protobuf (kubernetes/kubernetes#55541). Hence, we still can use unstruct type to handle CRDs which is already very useful in handling quite some use cases. For the built-in types, we probably have to enumerate all of them in case the protobuf is used. |
Thanks for your information, it's very helpful. I will think carefully about the relevant suggestions and update my proposal. |
Ⅰ. Describe what this PR does
add proposal to enhance the caching ability of Yurthub, which is related to the issue: #162 , mainly include:
Ⅱ. Does this pull request fix one issue?
Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews