Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Processor] Simplify some logic to reduce unnecessary duplicated work. #501

Merged
merged 1 commit into from
Nov 14, 2024

Conversation

grao1991
Copy link
Contributor

@grao1991 grao1991 commented Sep 6, 2024

This PR is trying to move some common logic/type/constant to resources.rs, avoid duplicate/unnecessary parsing/deserialization logic when converting between generic write resource and a specific resource type.

The main motivation is to improve the performance, no other behavior change is expected.
To give an idea on the performance impact, when running simple token minting workload, the processing time of token_v2_processor is 5x less than before.

This PR only handles FA and TokenV2 resources, leave other resources like Coin and Token for future PRs.

@grao1991 grao1991 force-pushed the grao_opt_processor branch 10 times, most recently from 02c2573 to 22a7444 Compare September 10, 2024 04:44
@grao1991 grao1991 marked this pull request as ready for review September 10, 2024 04:44
Copy link
Collaborator

This looks really good! Did we test this? Also don't forget to deploy before merging. For testing, maybe @rtso and @yuunlimm can help? We should at least compare the before and after, but it could also be a great case for an integration test looking for diffs.

@grao1991
Copy link
Contributor Author

This looks really good! Did we test this? Also don't forget to deploy before merging. For testing, maybe @rtso and @yuunlimm can help? We should at least compare the before and after, but it could also be a great case for an integration test looking for diffs.

I don't have a good way to get a full coverage for everything. I only manually sampled some rows and didn't see any difference before and after.

@grao1991 grao1991 requested a review from rtso September 13, 2024 22:31
@rtso
Copy link
Collaborator

rtso commented Sep 18, 2024

The general code structure looks good to me, but I'm hesitant to stamp this until we do adequate testing because it's such a big change. We can either write some integration tests (@yuunlimm or @larry-aptos do either of you have bandwidth to write these tests?) or we can deploy to devnet and manually check each resource is indexed.

@grao1991 grao1991 force-pushed the grao_opt_processor branch 5 times, most recently from 6e55195 to d1ea0a4 Compare November 14, 2024 01:48
@grao1991
Copy link
Contributor Author

Rebased my PR on top of tests. PTAL @rtso @yuunlimm @larry-aptos

@grao1991 grao1991 enabled auto-merge (squash) November 14, 2024 01:50
if MoveResource::get_outer_type_from_write_resource(write_resource) != Self::type_str() {
return Ok(None);
}
Ok(Some(write_resource.try_into()?))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we log the txn_version and wsc_index in case this fails? That will help us debug errors in parsing logic

@rtso rtso requested a review from dermanyang November 14, 2024 17:14
@grao1991 grao1991 merged commit 0e21881 into main Nov 14, 2024
7 checks passed
@grao1991 grao1991 deleted the grao_opt_processor branch November 14, 2024 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants