[dag] split fetcher into service and helper struct

[ibalajiarun]

Description

This PR splits the DAGFetcher into service and core components. The service is needed for fetching nodes during normal operation and runs in its own task. The core is used for one time fetching of nodes, for example during state sync.

Test Plan

[ibalajiarun]

Current dependencies on/for this PR:

• main
  • PR [dag] split fetcher into service and helper struct #9830 👈
    • PR [dag][bugfix] reverse nodes before adding to dag store #9831
      • PR [dag] preliminary state sync implementation #9724
        • PR [dag] broadcast CertifiedNodeMsg with LedgerInfo #9968
          • PR [dag] dag rebootstrap #9967
            • PR [consensus] introduce CommitSignerProvider trait #10006
              • PR [consensus] epoch manager refactoring; dag is coming #9994
                • PR [dag] commit signer #10040
                  • PR [dag] store parent block info in adapter #10041
                  • PR [dag] simplify DAGNetworkSender impl #10042
                  • PR [dag] async message handler support #10055
                  • PR [consensus][dag] dag integration helpers #10056
                  • PR [dag] epoch manager integration; dag is here #10007
        • PR [dag] state sync good case unit test #9832

This comment was auto-generated by Graphite.

[zekun000]

what's the motivation of this nested struct? having a trait for better testing?

[sasha8]

Note that the todo for retry logic is still missing

[ibalajiarun]

what's the motivation of this nested struct? having a trait for better testing?

For state sync, we don't need the channels, etc. I can just call the inner struct and fetch in a blocking way. The trait helps with testing.

[github-actions]

✅ Forge suite compat success on aptos-node-v1.6.2 ==> 48214b5bf13dc8c18af6ae976d9212979d7a6032

Compatibility test results for aptos-node-v1.6.2 ==> 48214b5bf13dc8c18af6ae976d9212979d7a6032 (PR)
1. Check liveness of validators at old version: aptos-node-v1.6.2
compatibility::simple-validator-upgrade::liveness-check : committed: 4746 txn/s, latency: 6656 ms, (p50: 6500 ms, p90: 9900 ms, p99: 15400 ms), latency samples: 189840
2. Upgrading first Validator to new version: 48214b5bf13dc8c18af6ae976d9212979d7a6032
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1791 txn/s, latency: 16366 ms, (p50: 18900 ms, p90: 22300 ms, p99: 22500 ms), latency samples: 93140
3. Upgrading rest of first batch to new version: 48214b5bf13dc8c18af6ae976d9212979d7a6032
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1848 txn/s, latency: 15634 ms, (p50: 18400 ms, p90: 22000 ms, p99: 22300 ms), latency samples: 92440
4. upgrading second batch to new version: 48214b5bf13dc8c18af6ae976d9212979d7a6032
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3084 txn/s, latency: 9858 ms, (p50: 10500 ms, p90: 12600 ms, p99: 13400 ms), latency samples: 129540
5. check swarm health
Compatibility test for aptos-node-v1.6.2 ==> 48214b5bf13dc8c18af6ae976d9212979d7a6032 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on aptos-node-v1.5.1 ==> 48214b5bf13dc8c18af6ae976d9212979d7a6032

Compatibility test results for aptos-node-v1.5.1 ==> 48214b5bf13dc8c18af6ae976d9212979d7a6032 (PR)
Upgrade the nodes to version: 48214b5bf13dc8c18af6ae976d9212979d7a6032
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 4945 txn/s, latency: 6466 ms, (p50: 6300 ms, p90: 9700 ms, p99: 13300 ms), latency samples: 182980
5. check swarm health
Compatibility test for aptos-node-v1.5.1 ==> 48214b5bf13dc8c18af6ae976d9212979d7a6032 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite realistic_env_max_load success on 48214b5bf13dc8c18af6ae976d9212979d7a6032

two traffics test: inner traffic : committed: 6716 txn/s, latency: 5845 ms, (p50: 5700 ms, p90: 7500 ms, p99: 11500 ms), latency samples: 2901740
two traffics test : committed: 100 txn/s, latency: 3147 ms, (p50: 3100 ms, p90: 3800 ms, p99: 4400 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.216, avg: 0.213", "QsPosToProposal: max: 0.156, avg: 0.142", "ConsensusProposalToOrdered: max: 0.602, avg: 0.561", "ConsensusOrderedToCommit: max: 0.520, avg: 0.493", "ConsensusProposalToCommit: max: 1.080, avg: 1.054"]
Max round gap was 1 [limit 4] at version 1447528. Max no progress secs was 3.852291 [limit 10] at version 1447528.
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking