[qs] another refactor

[zekun000]

This commit unifies fragment and proof processing such that we don't differentiate whether the message is for local or remote.

[sasha8]

I think we missed a possible attack. I can make you store incremental signatures for all digests in the system because I can lie about the batch_author in the signed digest. I think the fix is to check signed_digest.info().batch_author against what we store. Probably require an API change to batch_reader.exists.

[gelash]

but in order to check we should be storing it, no? so I guess that's why the original algorithm initialized the proofs from self (assuming self is and should be faster) and then appended matching digest/signatures - and I guess that also explains the check of the own signature, but in this case it should have been an assert.

I think that's a bit nicer than trying to check the author - as long as I'm not mistaken and checking the author almost implies that checking own signature would also be okay.

[gelash]

@igor-aptos since you found the self signature checks - what do you think?

[sasha8]

Why do you still call self.expire() in batch_generato.rs?

[github-actions]

✅ Forge suite land_blocking success on 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa

performance benchmark with full nodes : 5810 TPS, 6823 ms latency, 13900 ms p99 latency,(!) expired 320 out of 2481360 txns
Test Ok

• Grafana dashboard
• Humio Logs
• Pyroscope Profiling
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 8106 TPS, 4707 ms latency, 6500 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa
compatibility::simple-validator-upgrade::single-validator-upgrade : 4771 TPS, 8489 ms latency, 10800 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa
compatibility::simple-validator-upgrade::half-validator-upgrade : 5195 TPS, 7448 ms latency, 10000 ms p99 latency,no expired txns
4. upgrading second batch to new version: 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6997 TPS, 5498 ms latency, 9400 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa passed
Test Ok

• Grafana dashboard
• Humio Logs
• Pyroscope Profiling
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on cb4ba0a57c998c60cbab65af31a64875d2588ca5 ==> 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa

Compatibility test results for cb4ba0a57c998c60cbab65af31a64875d2588ca5 ==> 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa (PR)
Upgrade the nodes to version: 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa
framework_upgrade::framework-upgrade::full-framework-upgrade : 6825 TPS, 5719 ms latency, 8500 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for cb4ba0a57c998c60cbab65af31a64875d2588ca5 ==> 6807fde14f2bd66c9a4f4487ab0fb79870db0ffa passed
Test Ok

• Grafana dashboard
• Humio Logs
• Pyroscope Profiling
• Test runner output
• Test run is land-blocking

[sasha8]

Can we merge with the if in line 114? => if batch_author != self.peer_id {}