Add test checkpoint data builder #20749

lxfind · 2024-12-31T16:35:59Z

Description

This PR adds a builder that could generate arbitrary checkpoint data.
It will make it easy to test the indexer.

Test plan

Added unit tests.

Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

vercel · 2024-12-31T16:36:04Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
sui-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 4, 2025 7:56pm

3 Skipped Deployments

Name	Status	Preview	Updated (UTC)
multisig-toolkit	⬜️ Ignored (Inspect)	Visit Preview	Jan 4, 2025 7:56pm
sui-kiosk	⬜️ Ignored (Inspect)	Visit Preview	Jan 4, 2025 7:56pm
sui-typescript-docs	⬜️ Ignored (Inspect)	Visit Preview	Jan 4, 2025 7:56pm

wlmyng

I'm probably assuming a lot, but it looks like the TestCheckpointDataBuilder's purpose is to quickly generate transactions + checkpoints that are self-contained and individually correct, for the purpose of benchmarking indexer performance?

A noob question, but why not use Simulacrum? With #20729 approved, we could also generate test checkpoint data by writing test files through the transactional test runner, which would similarly allow us to generate arbitrary checkpoint data

wlmyng · 2025-01-02T17:51:42Z

crates/sui-types/src/test_checkpoint_data_builder.rs

+    pub fn new(checkpoint: u64) -> Self {
+        Self {
+            checkpoint,
+            epoch: 0,


Instead of defaulting to 0, thinking we should instead set it to None, and on TestCheckpointDataBuilder::build_checkpoint, if we didn't explicitly set this, then we'd error out

Otherwise, during our testing, we might neglect to set the correct epoch, resulting in strange behavior in our indexer

Perhaps a struct that wraps the checkpoint data builder, and would be solely responsible for advancing the epoch correctly

Edit: returning to this, seeing as how state isn't maintained across transactions and across checkpoints, I see that this isn't meant to be something like simulacrum to emulate a lockstep network but a way to generate txs and checkpoints in bulk

wlmyng · 2025-01-02T18:02:49Z

crates/sui-types/src/test_checkpoint_data_builder.rs

+    /// Mutate an existing object in the transaction.
+    /// `object_idx` is a convenient representation of the object's ID.
+    pub fn mutate_object(mut self, object_idx: u64) -> Self {
+        let tx_builder = self.next_transaction.as_mut().unwrap();
+        let object_id = derive_object_id(object_idx);
+        let object = self
+            .object_map
+            .get(&object_id)
+            .cloned()
+            .expect("Mutating an object that doesn't exist");
+        tx_builder.mutated_objects.push(object);
+        self
+    }


it looks like this doesnt make any changes to the object, just registers it in the tx builder's mutated_objects, so i guess this would be useful when testing object mutation but we don't care what exactly, or when mocking complex object mutation behavior

That is correct!

wlmyng · 2025-01-02T18:06:25Z

crates/sui-types/src/test_checkpoint_data_builder.rs

+    /// Complete the current transaction and add it to the checkpoint.
+    pub fn finish_transaction(mut self) -> Self {
+        let TransactionBuilder {
+            sender_idx,
+            gas,
+            created_objects,
+            mutated_objects,
+            deleted_objects,
+            events,
+        } = self.next_transaction.take().unwrap();


so within the checkpoint builder, state isn't preserved from one transaction to the next, we consume the contents of the transactionBuilder and finalize it into a standalone transaction

Just updated the PR to support generating multiple checkpoints.

lxfind · 2025-01-02T18:37:53Z

This will be used to write unit tests for various handler pipelines in indexer-alt, not for benchmarking.
It would be very difficult to use Simulacrum to quickly generate various shaped checkpoint data to write unittests. The test would be very complicated and mostly around how to construct certain transactions. For instance, to delete or wrap objects, or to test events, you would have to then implement Move contracts and make proper calls to do that.

lxfind · 2025-01-03T00:13:06Z

#20761 is a good demonstration on how much easier it becomes to test the handlers.

wlmyng

Gotcha, this looks good to me. This would indeed be nice to have for indexer unit tests, where we don't care so much about correctness from the network and more about whether we're indexing to the data or feature tables correctly

crates/sui-types/src/test_checkpoint_data_builder.rs

wlmyng · 2025-01-03T23:41:32Z

crates/sui-types/src/test_checkpoint_data_builder.rs

+    /// Wrap an existing object in the transaction.
+    /// `object_idx` is a convenient representation of the object's ID.
+    pub fn wrap_object(mut self, object_idx: u64) -> Self {
+        let tx_builder = self.checkpoint_builder.next_transaction.as_mut().unwrap();
+        let object_id = Self::derive_object_id(object_idx);
+        assert!(self.live_objects.contains_key(&object_id));
+        tx_builder.wrapped_objects.insert(object_id);
+        self
+    }


in this case, should we remove it from the live_objects set? and also update the mutated_objects set?

Returning to this: agree that it makes more sense to keep the various object functions restricted to its corresponding set, i.e, wrap and unwrap_object deal with the unwrapped_objects field, and so on, then consolidate in finish_transaction

crates/sui-types/src/test_checkpoint_data_builder.rs

wlmyng · 2025-01-03T23:50:38Z

crates/sui-types/src/test_checkpoint_data_builder.rs

+        self.live_objects
+            .extend(output_objects.iter().map(|o| (o.id(), o.clone())));


ah i see, when we unwrap, its added to output_objects, and then we extend the live_objects here

and this is because we derive the input_objects from live_objects, so if we do this earlier we'll erroneously add unwrapped objects

lxfind · 2025-01-04T02:09:12Z

@wlmyng Thank you for the review!

bmwill · 2025-01-04T20:46:58Z

What's the difference or benefit of this compared to simulacrum which already provides a way to build chain states

wlmyng · 2025-01-04T20:55:37Z

What's the difference or benefit of this compared to simulacrum which already provides a way to build chain states

Biggest thing for me over simulacrum is that while it simplifies the checkpoint and epoch creation process, there's still a lot of overhead to do even a simple transaction, and gets unwieldy when trying to emulate complex transaction effects

But perhaps we can consolidate? A simulacrum that can also mock out transaction execution .. or is that a thing already?

lxfind · 2025-01-04T21:43:43Z

What's the difference or benefit of this compared to simulacrum which already provides a way to build chain states

It's really about how easy it is to write tests that need some form of checkpoint data.
For instance, these tests: 833c798
I initially tried to implement it using simulacrum, but gave up fairly quickly. Most of the code is to deal with transaction construction in order to emit objects in the effects that I need. I also had to write quite some Move code for it. It would become harder once we start testing pipelines that need arbitrary Move calls, events and etc.

bmwill · 2025-01-04T22:02:31Z

Ah, so if i understand correctly then this doesn't do any execution but lets you just craft a txn/effect how you want to more easily create test scenarios that would otherwise take a lot of setup for?

lxfind · 2025-01-04T22:26:31Z

Correct!

lxfind temporarily deployed to sui-typescript-aws-kms-test-env December 31, 2024 16:36 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs December 31, 2024 16:36 View deployment

lxfind requested review from amnn, bmwill, emmazzz and wlmyng December 31, 2024 16:46

lxfind force-pushed the add-teset-checkpoint-data-builder branch from 9a95f7e to ba321ee Compare December 31, 2024 19:34

lxfind temporarily deployed to sui-typescript-aws-kms-test-env December 31, 2024 19:34 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs December 31, 2024 19:35 View deployment

lxfind force-pushed the add-teset-checkpoint-data-builder branch from ba321ee to 094faa4 Compare December 31, 2024 21:52

lxfind temporarily deployed to sui-typescript-aws-kms-test-env December 31, 2024 21:52 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs December 31, 2024 21:53 View deployment

wlmyng reviewed Jan 2, 2025

View reviewed changes

lxfind force-pushed the add-teset-checkpoint-data-builder branch from 094faa4 to cef22ed Compare January 2, 2025 19:28

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 2, 2025 19:28 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs January 2, 2025 19:29 View deployment

lxfind force-pushed the add-teset-checkpoint-data-builder branch from cef22ed to dc2470b Compare January 2, 2025 19:30

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 2, 2025 19:30 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs January 2, 2025 19:32 View deployment

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 2, 2025 21:21 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs January 2, 2025 21:22 View deployment

lxfind force-pushed the add-teset-checkpoint-data-builder branch from 6f9aa10 to 194ab05 Compare January 2, 2025 22:12

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 2, 2025 22:12 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs January 2, 2025 22:13 View deployment

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 3, 2025 17:42 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs January 3, 2025 17:43 View deployment

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 3, 2025 20:37 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs January 3, 2025 20:38 View deployment

wlmyng approved these changes Jan 3, 2025

View reviewed changes

lxfind added 6 commits January 3, 2025 18:09

Add test checkpoint data builder

1fae200

Add support for shared/immutable objects

a46bf32

Add ability to do move calls

7079c7d

Add support to change arbitrary object owner

ab0f29a

Move helper functions to struct methods

0981a53

Add comments

cbee358

lxfind force-pushed the add-teset-checkpoint-data-builder branch from 7b77e36 to cbee358 Compare January 4, 2025 02:20

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 4, 2025 02:20 — with GitHub Actions Inactive

lxfind enabled auto-merge (squash) January 4, 2025 02:20

vercel bot deployed to Preview – sui-docs January 4, 2025 02:21 View deployment

lxfind disabled auto-merge January 4, 2025 19:53

Fix simtest

7df7ca4

lxfind temporarily deployed to sui-typescript-aws-kms-test-env January 4, 2025 19:54 — with GitHub Actions Inactive

vercel bot deployed to Preview – sui-docs January 4, 2025 19:56 View deployment

lxfind merged commit 525f26f into main Jan 4, 2025
52 checks passed

lxfind deleted the add-teset-checkpoint-data-builder branch January 4, 2025 23:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test checkpoint data builder #20749

Add test checkpoint data builder #20749

lxfind commented Dec 31, 2024

vercel bot commented Dec 31, 2024 •

edited

Loading

wlmyng left a comment

wlmyng Jan 2, 2025

wlmyng Jan 2, 2025

lxfind Jan 2, 2025

wlmyng Jan 2, 2025

lxfind Jan 2, 2025

lxfind commented Jan 2, 2025

lxfind commented Jan 3, 2025

wlmyng left a comment

wlmyng Jan 3, 2025

wlmyng Jan 3, 2025 •

edited

Loading

wlmyng Jan 3, 2025

wlmyng Jan 3, 2025

lxfind commented Jan 4, 2025

bmwill commented Jan 4, 2025

wlmyng commented Jan 4, 2025

lxfind commented Jan 4, 2025

bmwill commented Jan 4, 2025

lxfind commented Jan 4, 2025

		self.live_objects
		.extend(output_objects.iter().map(\|o\| (o.id(), o.clone())));

Add test checkpoint data builder #20749

Add test checkpoint data builder #20749

Conversation

lxfind commented Dec 31, 2024

Description

Test plan

Release notes

vercel bot commented Dec 31, 2024 • edited Loading

wlmyng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lxfind commented Jan 2, 2025

lxfind commented Jan 3, 2025

wlmyng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wlmyng Jan 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lxfind commented Jan 4, 2025

bmwill commented Jan 4, 2025

wlmyng commented Jan 4, 2025

lxfind commented Jan 4, 2025

bmwill commented Jan 4, 2025

lxfind commented Jan 4, 2025

vercel bot commented Dec 31, 2024 •

edited

Loading

wlmyng Jan 3, 2025 •

edited

Loading