[consensus] payload manager enum to trait refactor

[ibalajiarun]

Description

Refactors PayloadManager such that it's now a trait implementation rather than an Enum. With a trait, we can use specific implementations of PayloadManager trait instead of matching and handling enum values in each impl methods.

With the new TPayloadManager trait, we now have the following implementations:

• DirectMempoolPayloadManager
• QuorumStorePayloadManager
• ConsensusObserverPayloadManager.

This setup should make it easier to write tests for PayloadManager and other components that use it.

[trunk-io]

⏱️ 8h 23m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
forge-e2e-test / forge	2h 30m	🟩 🟩 🟩 🟩 🟩 (+5 more)
test-fuzzers	2h 28m	🟥 🟥 🟩
test-target-determinator	56m	🟩 🟩 🟩 🟩 🟩 (+7 more)
forge-framework-upgrade-test / forge	17m	🟩
general-lints	16m	🟩 🟩 🟩 🟩 🟩 (+4 more)
rust-cargo-deny	16m	🟩 🟩 🟩 🟩 🟩 (+4 more)
forge-compat-test / forge	14m	🟩
check-dynamic-deps	13m	🟩 🟩 🟩 🟩 🟩 (+6 more)
rust-move-tests	7m	🟩
rust-move-tests	7m	🟩
rust-move-tests	7m	🟩
rust-move-tests	6m	🟩
rust-doc-tests	5m	🟩
execution-performance / test-target-determinator	4m	🟩
semgrep/ci	4m	🟩 🟩 🟩 🟩 🟩 (+6 more)
check	4m	🟩
rust-move-tests	3m	🟩
rust-move-tests	3m	🟩
rust-move-tests	3m	🟩
rust-move-tests	3m	🟩
rust-move-tests	3m	🟩
file_change_determinator	3m	🟩 🟩 🟥 🟩 🟩 (+6 more)
file_change_determinator	2m	🟩 🟩 🟩 🟩 🟩 (+7 more)
file_change_determinator	2m	🟩 🟩 🟩 🟩 🟩 (+7 more)
rust-move-tests	2m	🟩
permission-check	43s	🟩 🟩 🟩 🟩 🟩 (+7 more)
permission-check	36s	🟩 🟩 🟩 🟩 🟩 (+8 more)
permission-check	36s	🟩 🟩 🟩 🟩 🟩 (+8 more)
permission-check	34s	🟩 🟩 🟩 🟩 🟩 (+8 more)
permission-check	33s	🟩 🟩 🟩 🟩 🟩 (+8 more)
determine-docker-build-metadata	23s	🟩 🟩 🟩 🟩 🟩 (+7 more)
Backport PR	19s	🟥 🟥
execution-performance / single-node-performance	9s	🟩
permission-check	4s	🟩 🟩
rust-move-tests	1s	⬜

🚨 2 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
test-fuzzers	45m	37m
execution-performance / single-node-performance	9s	18m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[sasha8]

PR LGTM, but one comment on old code.

[sasha8]

Is there a race here?

[github-actions]

✅ Forge suite compat success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 459d30c272294456a122e87b7336ebafb0231d19

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 459d30c272294456a122e87b7336ebafb0231d19 (PR)
1. Check liveness of validators at old version: 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5
compatibility::simple-validator-upgrade::liveness-check : committed: 7249.934492176816 txn/s, latency: 4257.146949567909 ms, (p50: 3600 ms, p90: 5200 ms, p99: 28000 ms), latency samples: 310120
2. Upgrading first Validator to new version: 459d30c272294456a122e87b7336ebafb0231d19
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6189.094541501272 txn/s, latency: 4449.589393171062 ms, (p50: 5000 ms, p90: 5500 ms, p99: 5600 ms), latency samples: 118320
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7090.858127355418 txn/s, latency: 4631.741665287889 ms, (p50: 4700 ms, p90: 5400 ms, p99: 6000 ms), latency samples: 241760
3. Upgrading rest of first batch to new version: 459d30c272294456a122e87b7336ebafb0231d19
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 3156.654543440584 txn/s, latency: 8789.293312693499 ms, (p50: 9700 ms, p90: 11300 ms, p99: 11500 ms), latency samples: 64600
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6669.671378163116 txn/s, latency: 4682.730118144188 ms, (p50: 4800 ms, p90: 6600 ms, p99: 7000 ms), latency samples: 231920
4. upgrading second batch to new version: 459d30c272294456a122e87b7336ebafb0231d19
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 10429.562849675345 txn/s, latency: 2648.756695578591 ms, (p50: 2800 ms, p90: 3700 ms, p99: 5600 ms), latency samples: 189080
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10051.985227230689 txn/s, latency: 3358.162023202138 ms, (p50: 3300 ms, p90: 4000 ms, p99: 4900 ms), latency samples: 329280
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 459d30c272294456a122e87b7336ebafb0231d19 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite realistic_env_max_load success on 459d30c272294456a122e87b7336ebafb0231d19

two traffics test: inner traffic : committed: 9329.868518485724 txn/s, submitted: 9486.896384037951 txn/s, failed submission: 7.429801363096511 txn/s, expired: 157.02786555222664 txn/s, latency: 4911.170264646138 ms, (p50: 4200 ms, p90: 7900 ms, p99: 13300 ms), latency samples: 3547454
two traffics test : committed: 99.94220669279633 txn/s, latency: 2040.1295 ms, (p50: 2000 ms, p90: 2200 ms, p99: 8500 ms), latency samples: 2000
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.238, avg: 0.214", "QsPosToProposal: max: 1.357, avg: 0.969", "ConsensusProposalToOrdered: max: 0.324, avg: 0.295", "ConsensusOrderedToCommit: max: 0.421, avg: 0.398", "ConsensusProposalToCommit: max: 0.712, avg: 0.692"]
Max round gap was 1 [limit 4] at version 1821194. Max no progress secs was 5.820202 [limit 15] at version 1821194.
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 459d30c272294456a122e87b7336ebafb0231d19

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 459d30c272294456a122e87b7336ebafb0231d19 (PR)
Upgrade the nodes to version: 459d30c272294456a122e87b7336ebafb0231d19
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1131.0990470229171 txn/s, submitted: 1134.727271712646 txn/s, failed submission: 3.6282246897286843 txn/s, expired: 3.6282246897286843 txn/s, latency: 2861.230603448276 ms, (p50: 2100 ms, p90: 4800 ms, p99: 12300 ms), latency samples: 99760
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1098.813944967376 txn/s, submitted: 1100.6626329925577 txn/s, failed submission: 1.8486880251817053 txn/s, expired: 1.8486880251817053 txn/s, latency: 2847.607770767613 ms, (p50: 2100 ms, p90: 4800 ms, p99: 11100 ms), latency samples: 95100
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> 459d30c272294456a122e87b7336ebafb0231d19 passed
Upgrade the remaining nodes to version: 459d30c272294456a122e87b7336ebafb0231d19
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1203.3309434403923 txn/s, submitted: 1204.9738762674028 txn/s, failed submission: 1.642932827010483 txn/s, expired: 1.642932827010483 txn/s, latency: 2719.4378681490152 ms, (p50: 2100 ms, p90: 4500 ms, p99: 10300 ms), latency samples: 102540
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking