[ProjectTracking] [State Sync] Improvements

[VanBarbascu]

Goals

Background

We have delivered state sync from external storage and it is functional. While working on other projects we may come up with ideas of optimizing the current flow to enable other features.

Why should NEAR One work on this

Each optimization or bugfix will be accompanied by the explanation why is it needed.

What needs to be accomplished

Stabilize state sync.

Main use case

TODO

Links to external documentations and discussions

TODO

Estimated effort

TODO

Assumptions

TODO

Pre-requisites

N/A

Out of scope

N/A

Task list:

External storage state dumpers

Beta Give feedback

1. Skip upload of state parts when the node transitions to another shard layout

State download

Beta Give feedback

1. [External storage sync] Read existing files before downloading the state.

DSS

Beta Give feedback

1. [DSS] [State Sync] Update response messages to include bitmap of parts #9627
   C-good-first-issue Near Core Node +3
2. [DSS] Move downloading logic from Client Actor to StateSync Actor #9625
   2 of 5
   Near Core Node +2
   
3. [State Sync] [DSS only] Breakdown the state sync header into requests to different peers that track that shard
4. [State Sync] Create separate DB for State snapshots #10457
   Near Core Node +2