Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#190] Fix(core): Create serde class lazily #192

Merged
merged 6 commits into from
Aug 10, 2023
Merged

Conversation

mchades
Copy link
Contributor

@mchades mchades commented Aug 8, 2023

What changes were proposed in this pull request?

Lazily create the serde class in the classloader when serializing and deserializing

Why are the changes needed?

Class Not Found Exception will be thrown when serializing or deserializing catalog entity.

Fix: #190

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing UTs

@mchades mchades requested a review from jerryshao August 8, 2023 11:29
@github-actions
Copy link

github-actions bot commented Aug 8, 2023

Code Coverage Report

Overall Project 59% -0.28% 🟢
Files changed 58.52% 🔴

Module Coverage
core 67.81% -0.86% 🔴
Files
Module File Coverage
core ProtoEntitySerDe.java 62.17% -31.74% 🔴

@mchades mchades requested a review from yuqi1129 August 8, 2023 11:59
@yuqi1129
Copy link
Contributor

yuqi1129 commented Aug 9, 2023

LGTM

yuqi1129
yuqi1129 previously approved these changes Aug 9, 2023
Comment on lines 61 to 73
Class<? extends Message> protoClass =
entityToProto.computeIfAbsent(
clazz,
k -> {
try {
return (Class<? extends Message>)
loadClass(
ENTITY_TO_PROTO.get(k.getCanonicalName()), getClass().getClassLoader());
} catch (Exception e) {
throw new RuntimeException(
"Failed to create proto class " + k.getCanonicalName(), e);
}
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should make sure to put this code in a lock to avoid race conditions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any problem with loading or putting the same class twice ?

throw new RuntimeException(
"Failed to instantiate serde class " + entityClazz.getCanonicalName(), e);
}
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here.


ProtoSerDe<T, M> protoSerDe = (ProtoSerDe<T, M>) entityToSerDe.get(entityClass);
return protoSerDe.deserialize(m);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we remove this method, this method is pair with toProto, we'd better keep this.

Copy link
Contributor Author

@mchades mchades Aug 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toProto and fromProto is only the intermediate step of serialize and deserialize at present, not called elsewhere, so I make toProto private, and remove fromProto.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, fromProto method need to add a <T extends Entity> T parameter as there may be one proto maps to multiple entity later on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make pair with toProto, already add fromProto back and modified method signature

@@ -86,52 +52,77 @@ public <T extends Entity> byte[] serialize(T t) throws IOException {
public <T extends Entity> T deserialize(byte[] bytes, Class<T> clazz, ClassLoader classLoader)
throws IOException {
Any any = Any.parseFrom(bytes);
Class<? extends Message> protoClass =
getProtoClass(clazz, Thread.currentThread().getContextClassLoader());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use the classLoader provided in method parameter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

private <T extends Entity, M extends Message> M toProto(T t) throws IOException {
ProtoSerDe<T, M> protoSerDe =
(ProtoSerDe<T, M>)
getProtoSerde(t.getClass(), Thread.currentThread().getContextClassLoader());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should take classloader as a method parameter and pass in from the caller of this method.

We'd better to make sure classloader is got only from one place and pass in to different methods, so that we could make sure all classloader is the same and change one place is enough (if we want to use different classloader).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

private <T extends Entity, M extends Message> T fromProto(M m, Class<T> entityClass)
throws IOException {
ProtoSerDe<T, Message> protoSerDe =
getProtoSerde(entityClass, Thread.currentThread().getContextClassLoader());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Contributor

@jerryshao jerryshao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@jerryshao jerryshao merged commit 46b1cb6 into apache:main Aug 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement] Class Not Found Exception when creating serde object
3 participants