[dag] state sync good case unit test #9832

ibalajiarun · 2023-08-30T05:15:03Z

Description

Test Plan

ibalajiarun · 2023-08-30T05:15:07Z

Current dependencies on/for this PR:

main
- PR [dag] split fetcher into service and helper struct #9830
  - PR [dag][bugfix] reverse nodes before adding to dag store #9831
    - PR [dag] preliminary state sync implementation #9724
      - PR [dag] broadcast CertifiedNodeMsg with LedgerInfo #9968
        
        PR [dag] dag rebootstrap #9967
        
        PR [consensus] introduce CommitSignerProvider trait #10006
        
        PR [consensus] epoch manager refactoring; dag is coming #9994
        
        PR [dag] commit signer #10040
        
        PR [dag] store parent block info in adapter #10041
        PR [dag] simplify DAGNetworkSender impl #10042
        PR [dag] async message handler support #10055
        PR [consensus][dag] dag integration helpers #10056
        PR [dag] epoch manager integration; dag is here #10007
      - PR [dag] state sync good case unit test #9832 👈

This comment was auto-generated by Graphite.

zekun000 · 2023-09-08T21:19:53Z

consensus/src/dag/tests/dag_state_sync_tests.rs

+        &self,
+        remote_request: RemoteFetchRequest,
+        _responders: Vec<Author>,
+        new_dag: Arc<RwLock<Dag>>,


nit: I think when we do fetch it can be a Dag without arc mutex?

yes, but the same trait is used for fetcher service too, which needs to pass an arc mutex.

zekun000 · 2023-09-08T21:22:49Z

consensus/src/dag/tests/dag_state_sync_tests.rs

+    }
+    let slow_dag = Arc::new(RwLock::new(slow_dag));
+
+    let li_node = nodes[LI_ROUNDS - 1].first().unwrap().clone().unwrap();


just for zero indexing.

sasha8

stamp

sasha8

stamp

github-actions · 2023-09-09T20:54:09Z

✅ Forge suite `compat` success on `aptos-node-v1.6.2` ==> `de2cc9e1f492eab890386cc35ccdb4517ac61571`

Compatibility test results for aptos-node-v1.6.2 ==> de2cc9e1f492eab890386cc35ccdb4517ac61571 (PR)
1. Check liveness of validators at old version: aptos-node-v1.6.2
compatibility::simple-validator-upgrade::liveness-check : committed: 4101 txn/s, latency: 6963 ms, (p50: 6600 ms, p90: 9900 ms, p99: 25300 ms), latency samples: 172280
2. Upgrading first Validator to new version: de2cc9e1f492eab890386cc35ccdb4517ac61571
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1854 txn/s, latency: 15640 ms, (p50: 18600 ms, p90: 22000 ms, p99: 22300 ms), latency samples: 92700
3. Upgrading rest of first batch to new version: de2cc9e1f492eab890386cc35ccdb4517ac61571
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1352 txn/s, submitted: 1393 txn/s, expired: 41 txn/s, latency: 15813 ms, (p50: 17400 ms, p90: 21600 ms, p99: 22000 ms), latency samples: 85188
4. upgrading second batch to new version: de2cc9e1f492eab890386cc35ccdb4517ac61571
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3377 txn/s, latency: 9274 ms, (p50: 9900 ms, p90: 13200 ms, p99: 13900 ms), latency samples: 138480
5. check swarm health
Compatibility test for aptos-node-v1.6.2 ==> de2cc9e1f492eab890386cc35ccdb4517ac61571 passed
Test Ok

Grafana dashboard
Humio Logs
Test runner output
Test run is land-blocking

github-actions · 2023-09-09T21:01:07Z

✅ Forge suite `realistic_env_max_load` success on `de2cc9e1f492eab890386cc35ccdb4517ac61571`

two traffics test: inner traffic : committed: 6159 txn/s, latency: 6360 ms, (p50: 6000 ms, p90: 8400 ms, p99: 15400 ms), latency samples: 2667120
two traffics test : committed: 100 txn/s, latency: 3084 ms, (p50: 3000 ms, p90: 4100 ms, p99: 6900 ms), latency samples: 1900
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.261, avg: 0.216", "QsPosToProposal: max: 0.386, avg: 0.184", "ConsensusProposalToOrdered: max: 0.649, avg: 0.623", "ConsensusOrderedToCommit: max: 0.547, avg: 0.511", "ConsensusProposalToCommit: max: 1.177, avg: 1.134"]
Max round gap was 2 [limit 4] at version 2282537. Max no progress secs was 7.577653 [limit 10] at version 2282537.
Test Ok

Grafana dashboard
Humio Logs
Test runner output
Test run is land-blocking

github-actions · 2023-09-09T21:09:41Z

✅ Forge suite `framework_upgrade` success on `aptos-node-v1.5.1` ==> `de2cc9e1f492eab890386cc35ccdb4517ac61571`

Compatibility test results for aptos-node-v1.5.1 ==> de2cc9e1f492eab890386cc35ccdb4517ac61571 (PR)
Upgrade the nodes to version: de2cc9e1f492eab890386cc35ccdb4517ac61571
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 3429 txn/s, latency: 6223 ms, (p50: 6300 ms, p90: 9300 ms, p99: 11700 ms), latency samples: 192060
5. check swarm health
Compatibility test for aptos-node-v1.5.1 ==> de2cc9e1f492eab890386cc35ccdb4517ac61571 passed
Test Ok

Grafana dashboard
Humio Logs
Test runner output
Test run is land-blocking

ibalajiarun requested a review from zekun000 as a code owner August 30, 2023 05:15

This was referenced Aug 30, 2023

[dag] split fetcher into service and helper struct #9830

Merged

[dag][bugfix] reverse nodes before adding to dag store #9831

Merged

ibalajiarun mentioned this pull request Aug 30, 2023

[dag] preliminary state sync implementation #9724

Merged

ibalajiarun requested a review from sasha8 August 30, 2023 05:17

ibalajiarun force-pushed the balaji/dag-state-sync branch from 023b879 to 977b9b1 Compare September 1, 2023 21:05

ibalajiarun force-pushed the balaji/dag-state-sync-test branch from 2505731 to abfaaa9 Compare September 1, 2023 21:05

ibalajiarun force-pushed the balaji/dag-state-sync branch 6 times, most recently from 2f4d37d to ae30b43 Compare September 5, 2023 01:06

ibalajiarun force-pushed the balaji/dag-state-sync-test branch 3 times, most recently from c094f89 to 215f269 Compare September 5, 2023 20:43

ibalajiarun force-pushed the balaji/dag-state-sync branch from ae30b43 to 1a405d8 Compare September 5, 2023 20:47

ibalajiarun force-pushed the balaji/dag-state-sync-test branch from 215f269 to e275b22 Compare September 5, 2023 20:48

ibalajiarun force-pushed the balaji/dag-state-sync branch 2 times, most recently from efc7ec3 to 7e41139 Compare September 6, 2023 00:07

ibalajiarun mentioned this pull request Sep 7, 2023

[dag] dag rebootstrap #9967

Merged

ibalajiarun force-pushed the balaji/dag-state-sync branch from 7e41139 to 3932158 Compare September 7, 2023 21:48

ibalajiarun mentioned this pull request Sep 7, 2023

[dag] broadcast CertifiedNodeMsg with LedgerInfo #9968

Merged

ibalajiarun force-pushed the balaji/dag-state-sync branch 2 times, most recently from fab0ad8 to 6dd6796 Compare September 7, 2023 23:41

ibalajiarun force-pushed the balaji/dag-state-sync-test branch from e275b22 to 5211cfe Compare September 7, 2023 23:41

Base automatically changed from balaji/dag-state-sync to main September 8, 2023 14:45

zekun000 approved these changes Sep 8, 2023

View reviewed changes

sasha8 reviewed Sep 8, 2023

View reviewed changes

sasha8 approved these changes Sep 8, 2023

View reviewed changes

ibalajiarun force-pushed the balaji/dag-state-sync-test branch from 5211cfe to 29b7644 Compare September 9, 2023 18:23

ibalajiarun enabled auto-merge (squash) September 9, 2023 18:25

This comment has been minimized.

Sign in to view

[dag] state sync good case unit test

de2cc9e

ibalajiarun force-pushed the balaji/dag-state-sync-test branch from 29b7644 to de2cc9e Compare September 9, 2023 20:25

This comment has been minimized.

Sign in to view

ibalajiarun merged commit b03ef3e into main Sep 9, 2023

ibalajiarun deleted the balaji/dag-state-sync-test branch September 9, 2023 21:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dag] state sync good case unit test #9832

[dag] state sync good case unit test #9832

ibalajiarun commented Aug 30, 2023

ibalajiarun commented Aug 30, 2023 •

edited

Loading

zekun000 Sep 8, 2023

ibalajiarun Sep 9, 2023

zekun000 Sep 8, 2023

ibalajiarun Sep 9, 2023

sasha8 left a comment

sasha8 left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Sep 9, 2023

github-actions bot commented Sep 9, 2023

github-actions bot commented Sep 9, 2023

[dag] state sync good case unit test #9832

[dag] state sync good case unit test #9832

Conversation

ibalajiarun commented Aug 30, 2023

Description

Test Plan

ibalajiarun commented Aug 30, 2023 • edited Loading

zekun000 Sep 8, 2023

Choose a reason for hiding this comment

ibalajiarun Sep 9, 2023

Choose a reason for hiding this comment

zekun000 Sep 8, 2023

Choose a reason for hiding this comment

ibalajiarun Sep 9, 2023

Choose a reason for hiding this comment

sasha8 left a comment

Choose a reason for hiding this comment

sasha8 left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Sep 9, 2023

✅ Forge suite compat success on aptos-node-v1.6.2 ==> de2cc9e1f492eab890386cc35ccdb4517ac61571

github-actions bot commented Sep 9, 2023

✅ Forge suite realistic_env_max_load success on de2cc9e1f492eab890386cc35ccdb4517ac61571

github-actions bot commented Sep 9, 2023

✅ Forge suite framework_upgrade success on aptos-node-v1.5.1 ==> de2cc9e1f492eab890386cc35ccdb4517ac61571

ibalajiarun commented Aug 30, 2023 •

edited

Loading

✅ Forge suite `compat` success on `aptos-node-v1.6.2` ==> `de2cc9e1f492eab890386cc35ccdb4517ac61571`

✅ Forge suite `realistic_env_max_load` success on `de2cc9e1f492eab890386cc35ccdb4517ac61571`

✅ Forge suite `framework_upgrade` success on `aptos-node-v1.5.1` ==> `de2cc9e1f492eab890386cc35ccdb4517ac61571`