[stateless_validaton] Create stub for PartialEncodedStateWitness based requests #11091

shreyan-gupta · 2024-04-17T00:26:35Z

Part 5

This PR creates stub functions for sending and receiving PartialEncodedStateWitness. The full details of the content of PartialEncodedStateWitness is still undecided and would be filled in in a separate PR.

We have two message types, PartialEncodedStateWitnessMessage and PartialEncodedStateWitnessForwardMessage

PartialEncodedStateWitnessMessage is used to send state parts from chunk producer to all the other chunk validators who are part owners
PartialEncodedStateWitnessForwardMessage receives the PartialEncodedStateWitnessMessage and forwards it to all other chunk validators.

NOTE: I couldn't create stacked PRs. Please review these commits only.

…to StateWitnessDistributionActor

…itnessActor

…tateWitnessActor

…d requests

codecov · 2024-04-17T00:52:17Z

Codecov Report

Attention: Patch coverage is 4.44444% with 43 lines in your changes are missing coverage. Please review.

Project coverage is 71.06%. Comparing base (2bbde59) to head (d0ad393).

Files	Patch %	Lines
...nt/src/stateless_validation/state_witness_actor.rs	0.00%	20 Missing ⚠️
.../src/stateless_validation/state_witness_actions.rs	9.09%	10 Missing ⚠️
chain/client/src/test_utils/test_loop.rs	0.00%	6 Missing ⚠️
chain/network/src/network_protocol/mod.rs	0.00%	2 Missing ⚠️
chain/network/src/peer/testonly.rs	0.00%	1 Missing and 1 partial ⚠️
chain/network/src/peer_manager/testonly.rs	0.00%	1 Missing and 1 partial ⚠️
core/primitives/src/stateless_validation.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #11091      +/-   ##
==========================================
- Coverage   71.07%   71.06%   -0.02%     
==========================================
  Files         767      767              
  Lines      153367   153390      +23     
  Branches   153367   153390      +23     
==========================================
  Hits       109005   109005              
- Misses      39909    39936      +27     
+ Partials     4453     4449       -4

Flag	Coverage Δ
backward-compatibility	`0.24% <0.00%> (-0.01%)`	⬇️
db-migration	`0.24% <0.00%> (-0.01%)`	⬇️
genesis-check	`1.43% <0.00%> (-0.01%)`	⬇️
integration-tests	`36.86% <4.44%> (+0.07%)`	⬆️
linux	`69.45% <2.22%> (-0.03%)`	⬇️
linux-nightly	`70.53% <4.44%> (-0.01%)`	⬇️
macos	`54.20% <2.43%> (-0.05%)`	⬇️
pytests	`1.66% <0.00%> (-0.01%)`	⬇️
sanity-checks	`1.45% <0.00%> (-0.01%)`	⬇️
unittests	`66.74% <2.22%> (-0.01%)`	⬇️
upgradability	`0.29% <0.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

saketh-are · 2024-04-17T05:40:17Z

chain/network/src/peer_manager/peer_manager_actor.rs

@@ -995,6 +995,31 @@ impl PeerManagerActor {
                );
                NetworkResponses::NoResponse
            }
+            NetworkRequests::PartialEncodedStateWitness(chunk_validators, partial_witness) => {
+                for chunk_validator in chunk_validators {
+                    self.state.send_message_to_account(


I think you just want one recipient here; each part should only be sent to the one validator which is responsible for forwarding it to other validators.

Right, yeah! will change. Thanks! In general, is it a better idea to send one network request with Vec<(validator, partial_witness)> or send multiple network requests with one (validator, partial_witness)

I don't see any disadvantages of batching the requests into a Vec. It's more efficient and the type is internal anyway (just passed between actix actors) so we can change it freely. Batching indicates more clearly what is happening in case we want to think in the direction of optimizing multicast on the network level in the future.

…ed_state_witness

We missed that when refactoring chunk endorsement logic. The old `ChunkEndorsement` message was removed in #12131, but we forgot to add the new message introduced in #11091. I've noticed that by looking at mainnet metrics where `VersionedChunkEndorsement` message is only sent via T2.

Shreyan Gupta added 11 commits April 15, 2024 17:36

[stateless_validation] Introduce new StateWitnessDistributionActor

5a41aa9

minor changes

38b98da

changes

4a46b2c

fix

e2636a3

fix

a8ece38

[stateless_validation] Shift handling of ChunkStateWitnessAckMessage …

25aa8f3

…to StateWitnessDistributionActor

fix

4edc2a4

clippy fix

f0471d6

[stateless_validation] Rename StateWitnessDistributionActor to StateW…

85d64ef

…itnessActor

[stateless_validation] Push get_chunk_validator_assignments call to S…

2c201fa

…tateWitnessActor

[stateless_validaton] Create stub for PartialEncodedStateWitness base…

a212d32

…d requests

shreyan-gupta requested review from tayfunelmas, pugachAG, saketh-are and robin-near April 17, 2024 00:26

shreyan-gupta requested a review from a team as a code owner April 17, 2024 00:26

shreyan-gupta added the A-stateless-validation Area: stateless validation label Apr 17, 2024

saketh-are approved these changes Apr 17, 2024

View reviewed changes

shreyan-gupta mentioned this pull request Apr 17, 2024

[stateless validation]: Implement state witness distribution by parts near/near-one-project-tracking#67

Closed

tayfunelmas approved these changes Apr 17, 2024

View reviewed changes

Shreyan Gupta added 2 commits April 17, 2024 21:59

Merge branch 'master' into shreyan/stateless_validation/partial_encod…

6d06037

…ed_state_witness

resolve comment

d0ad393

shreyan-gupta enabled auto-merge April 18, 2024 05:05

shreyan-gupta added this pull request to the merge queue Apr 18, 2024

Merged via the queue into near:master with commit 2a3e0b5 Apr 18, 2024
27 of 29 checks passed

shreyan-gupta deleted the shreyan/stateless_validation/partial_encoded_state_witness branch April 18, 2024 05:48

pugachAG mentioned this pull request Dec 16, 2024

fix: use T1 for sending chunk endorsements #12632

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[stateless_validaton] Create stub for PartialEncodedStateWitness based requests #11091

[stateless_validaton] Create stub for PartialEncodedStateWitness based requests #11091

shreyan-gupta commented Apr 17, 2024 •

edited

Loading

codecov bot commented Apr 17, 2024 •

edited

Loading

saketh-are Apr 17, 2024 •

edited

Loading

shreyan-gupta Apr 17, 2024

saketh-are Apr 17, 2024

[stateless_validaton] Create stub for PartialEncodedStateWitness based requests #11091

[stateless_validaton] Create stub for PartialEncodedStateWitness based requests #11091

Conversation

shreyan-gupta commented Apr 17, 2024 • edited Loading

codecov bot commented Apr 17, 2024 • edited Loading

Codecov Report

saketh-are Apr 17, 2024 • edited Loading

Choose a reason for hiding this comment

shreyan-gupta Apr 17, 2024

Choose a reason for hiding this comment

saketh-are Apr 17, 2024

Choose a reason for hiding this comment

shreyan-gupta commented Apr 17, 2024 •

edited

Loading

codecov bot commented Apr 17, 2024 •

edited

Loading

saketh-are Apr 17, 2024 •

edited

Loading