reconfig in progress errmsg

[zjma]

Description

More actionable error message when a txn is aborted due to in-progress async reconfig.

How Has This Been Tested?

n/a

Key Areas to Review

n/a

Type of Change

• Documentation update

Checklist

• I have read and followed the CONTRIBUTING doc
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I identified and added all stakeholders and component owners affected by this change as reviewers
• I tested both happy and unhappy path of the functionality
• I have made corresponding changes to the documentation

[trunk-io]

⏱️ 1h 30m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
execution-performance / single-node-performance	25m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
check-dynamic-deps	6m	🟩 🟩 🟩 🟩
rust-cargo-deny	5m	🟩 🟩 🟩
rust-doc-tests	5m	🟩
test-target-determinator	5m	🟩
execution-performance / test-target-determinator	5m	🟩
rust-move-tests	4m	⬜
check	4m	🟩
semgrep/ci	2m	🟩 🟩 🟩 🟩
general-lints	2m	🟩 🟩 🟩
fetch-last-released-docker-image-tag	1m	🟩
file_change_determinator	35s	🟩 🟩 ⬜ 🟩
file_change_determinator	11s	🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
execution-performance / single-node-performance	25m	15m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[github-actions]

✅ Forge suite realistic_env_max_load success on 89d681619a3abaa02b5ee7ee3915b8190429558a

two traffics test: inner traffic : committed: 14257.43 txn/s, latency: 2790.71 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3600 ms), latency samples: 5421040
two traffics test : committed: 100.01 txn/s, latency: 1522.44 ms, (p50: 1400 ms, p70: 1400, p90: 1600 ms, p99: 9300 ms), latency samples: 1780
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.016, avg: 1.581", "ConsensusProposalToOrdered: max: 0.323, avg: 0.296", "ConsensusOrderedToCommit: max: 0.378, avg: 0.363", "ConsensusProposalToCommit: max: 0.679, avg: 0.660"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.86s no progress at version 2575854 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.62s no progress at version 2575852 (avg 8.62s) [limit 15].
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite compat success on be3ccc58ef518259d63fab7f3e3613f2a6214268 ==> 89d681619a3abaa02b5ee7ee3915b8190429558a

Compatibility test results for be3ccc58ef518259d63fab7f3e3613f2a6214268 ==> 89d681619a3abaa02b5ee7ee3915b8190429558a (PR)
1. Check liveness of validators at old version: be3ccc58ef518259d63fab7f3e3613f2a6214268
compatibility::simple-validator-upgrade::liveness-check : committed: 14652.72 txn/s, latency: 2367.07 ms, (p50: 2100 ms, p70: 2100, p90: 2400 ms, p99: 7900 ms), latency samples: 552440
2. Upgrading first Validator to new version: 89d681619a3abaa02b5ee7ee3915b8190429558a
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7429.68 txn/s, latency: 3871.94 ms, (p50: 4300 ms, p70: 4600, p90: 4700 ms, p99: 4800 ms), latency samples: 139660
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6638.65 txn/s, latency: 4529.21 ms, (p50: 4700 ms, p70: 4800, p90: 4800 ms, p99: 6300 ms), latency samples: 247280
3. Upgrading rest of first batch to new version: 89d681619a3abaa02b5ee7ee3915b8190429558a
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 8386.54 txn/s, latency: 3422.00 ms, (p50: 3800 ms, p70: 4000, p90: 4100 ms, p99: 4200 ms), latency samples: 152820
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 8565.99 txn/s, latency: 3784.19 ms, (p50: 4000 ms, p70: 4100, p90: 4200 ms, p99: 4200 ms), latency samples: 283200
4. upgrading second batch to new version: 89d681619a3abaa02b5ee7ee3915b8190429558a
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 12942.07 txn/s, latency: 2065.21 ms, (p50: 1900 ms, p70: 2500, p90: 3000 ms, p99: 3100 ms), latency samples: 227120
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 8663.29 txn/s, latency: 3675.60 ms, (p50: 2000 ms, p70: 3000, p90: 3300 ms, p99: 19400 ms), latency samples: 407420
5. check swarm health
Compatibility test for be3ccc58ef518259d63fab7f3e3613f2a6214268 ==> 89d681619a3abaa02b5ee7ee3915b8190429558a passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on be3ccc58ef518259d63fab7f3e3613f2a6214268 ==> 89d681619a3abaa02b5ee7ee3915b8190429558a

Compatibility test results for be3ccc58ef518259d63fab7f3e3613f2a6214268 ==> 89d681619a3abaa02b5ee7ee3915b8190429558a (PR)
Upgrade the nodes to version: 89d681619a3abaa02b5ee7ee3915b8190429558a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1269.62 txn/s, submitted: 1272.73 txn/s, failed submission: 3.11 txn/s, expired: 3.11 txn/s, latency: 2523.04 ms, (p50: 2100 ms, p70: 2400, p90: 3900 ms, p99: 5000 ms), latency samples: 114360
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1462.50 txn/s, submitted: 1465.23 txn/s, failed submission: 2.73 txn/s, expired: 2.73 txn/s, latency: 2096.12 ms, (p50: 2100 ms, p70: 2400, p90: 3000 ms, p99: 4100 ms), latency samples: 128380
5. check swarm health
Compatibility test for be3ccc58ef518259d63fab7f3e3613f2a6214268 ==> 89d681619a3abaa02b5ee7ee3915b8190429558a passed
Upgrade the remaining nodes to version: 89d681619a3abaa02b5ee7ee3915b8190429558a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1431.92 txn/s, submitted: 1434.77 txn/s, failed submission: 2.85 txn/s, expired: 2.85 txn/s, latency: 2180.31 ms, (p50: 2100 ms, p70: 2400, p90: 3300 ms, p99: 4200 ms), latency samples: 130540
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking