Stop locking BufferedState for read

[msmouse]

Description

Move the part which need to be read out, so that the read path is not blocked by buffered state back pressure or sync commits.

Also the pre commited version is now in the same lock of the latest state, eliminating some edge cases.

How Has This Been Tested?

Type of Change

• New feature

Which Components or Systems Does This Change Impact?

• Validator Node

[trunk-io]

⏱️ 4h 18m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
execution-performance / single-node-performance	1h 32m	🟩 🟩 🟩 🟩
execution-performance / test-target-determinator	19m	🟩 🟩 🟩 🟩
test-target-determinator	18m	🟩 ⬜ 🟩 🟩
check	15m	🟩 ⬜ 🟩 🟩
check-dynamic-deps	14m	🟩 🟩 🟩 🟩 🟩 (+3 more)
rust-cargo-deny	13m	🟩 🟩 🟩 🟩 🟩 (+2 more)
rust-move-tests	10m	🟩
rust-move-tests	9m	🟩
rust-move-tests	9m	🟩
rust-move-tests	7m	⬜
rust-move-tests	7m	⬜
rust-move-tests	7m	⬜
fetch-last-released-docker-image-tag	7m	🟩 🟩 🟩 🟩
rust-doc-tests	5m	🟩
rust-doc-tests	5m	🟩

🚨 3 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
execution-performance / test-target-determinator	6m	4m
execution-performance / single-node-performance	25m	20m
test-target-determinator	5m	4m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[github-actions]

✅ Forge suite realistic_env_max_load success on fcca36464035750deaab5c33cfad0c39cce08195

two traffics test: inner traffic : committed: 13930.96 txn/s, latency: 2864.13 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 12700 ms), latency samples: 5296880
two traffics test : committed: 100.00 txn/s, latency: 1618.30 ms, (p50: 1400 ms, p70: 1400, p90: 1500 ms, p99: 11100 ms), latency samples: 1820
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.096, avg: 1.616", "ConsensusProposalToOrdered: max: 0.328, avg: 0.297", "ConsensusOrderedToCommit: max: 0.357, avg: 0.349", "ConsensusProposalToCommit: max: 0.656, avg: 0.646"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.02s no progress at version 45326 (avg 0.21s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.47s no progress at version 5708884 (avg 8.44s) [limit 15].
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on 8a161886956d0b8360b152f051628bed624f7c24 ==> fcca36464035750deaab5c33cfad0c39cce08195

Compatibility test results for 8a161886956d0b8360b152f051628bed624f7c24 ==> fcca36464035750deaab5c33cfad0c39cce08195 (PR)
Upgrade the nodes to version: fcca36464035750deaab5c33cfad0c39cce08195
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1433.04 txn/s, submitted: 1435.29 txn/s, failed submission: 2.25 txn/s, expired: 2.25 txn/s, latency: 2199.25 ms, (p50: 2100 ms, p70: 2400, p90: 3300 ms, p99: 3900 ms), latency samples: 127400
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1557.22 txn/s, submitted: 1558.97 txn/s, failed submission: 1.75 txn/s, expired: 1.75 txn/s, latency: 2182.57 ms, (p50: 2100 ms, p70: 2200, p90: 3300 ms, p99: 4800 ms), latency samples: 124340
5. check swarm health
Compatibility test for 8a161886956d0b8360b152f051628bed624f7c24 ==> fcca36464035750deaab5c33cfad0c39cce08195 passed
Upgrade the remaining nodes to version: fcca36464035750deaab5c33cfad0c39cce08195
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1273.47 txn/s, submitted: 1276.10 txn/s, failed submission: 2.63 txn/s, expired: 2.63 txn/s, latency: 2420.63 ms, (p50: 2100 ms, p70: 2400, p90: 3700 ms, p99: 5100 ms), latency samples: 116280
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite compat success on 8a161886956d0b8360b152f051628bed624f7c24 ==> fcca36464035750deaab5c33cfad0c39cce08195

Compatibility test results for 8a161886956d0b8360b152f051628bed624f7c24 ==> fcca36464035750deaab5c33cfad0c39cce08195 (PR)
1. Check liveness of validators at old version: 8a161886956d0b8360b152f051628bed624f7c24
compatibility::simple-validator-upgrade::liveness-check : committed: 17065.50 txn/s, latency: 1964.65 ms, (p50: 1800 ms, p70: 1900, p90: 2600 ms, p99: 6000 ms), latency samples: 559760
2. Upgrading first Validator to new version: fcca36464035750deaab5c33cfad0c39cce08195
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7799.13 txn/s, latency: 3708.84 ms, (p50: 4100 ms, p70: 4300, p90: 4400 ms, p99: 4500 ms), latency samples: 144540
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7223.77 txn/s, latency: 4406.85 ms, (p50: 4300 ms, p70: 4400, p90: 6800 ms, p99: 7200 ms), latency samples: 248840
3. Upgrading rest of first batch to new version: fcca36464035750deaab5c33cfad0c39cce08195
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7896.36 txn/s, latency: 3528.90 ms, (p50: 3900 ms, p70: 4200, p90: 4300 ms, p99: 4300 ms), latency samples: 145760
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7727.35 txn/s, latency: 4109.43 ms, (p50: 4200 ms, p70: 4400, p90: 6100 ms, p99: 6400 ms), latency samples: 258280
4. upgrading second batch to new version: fcca36464035750deaab5c33cfad0c39cce08195
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 12031.87 txn/s, latency: 2284.06 ms, (p50: 2400 ms, p70: 2500, p90: 2900 ms, p99: 3100 ms), latency samples: 208780
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 11592.10 txn/s, latency: 2684.23 ms, (p50: 2500 ms, p70: 2800, p90: 4000 ms, p99: 5500 ms), latency samples: 375960
5. check swarm health
Compatibility test for 8a161886956d0b8360b152f051628bed624f7c24 ==> fcca36464035750deaab5c33cfad0c39cce08195 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking