fix: during resharding, reassign outgoing receipts to lowest index child #9362
Conversation
wacban
force-pushed
the
waclaw-resharding-outgoing-receipts
branch
from
bf79a62
to
3b7ff0b
Compare
| }); | ||
| } | ||
| if block_header.height() != last_included_height { | ||
| receipts_block_hash = *block_header.prev_hash(); |
There was a problem hiding this comment.
@robin-near, just for my understanding, when is it possible for the prev_block_height to not be last_included_height? Is it when we have an orphan block chain?
There was a problem hiding this comment.
I believe it's when a block is missing a chunk. Then the last_included_height will be the latest block height where a chunk for the shard is present.
There was a problem hiding this comment.
Yeah, what @wacban said. It's one of the very cryptic parts of the codebase - chunks can be missing from blocks, and the logic to handle that is very complex.
| }); | ||
| } | ||
| if block_header.height() != last_included_height { | ||
| receipts_block_hash = *block_header.prev_hash(); |
There was a problem hiding this comment.
Yeah, what @wacban said. It's one of the very cryptic parts of the codebase - chunks can be missing from blocks, and the logic to handle that is very complex.
| /// 4' will get no outgoing receipts from its parent 3 | ||
| /// All receipts are distributed to children, each exactly once. | ||
| #[cfg(feature = "protocol_feature_simple_nightshade_v2")] | ||
| fn reassign_outgoing_receipts_for_resharding_v2( |
There was a problem hiding this comment.
How does this handle when split shards are in the middle of shard IDs, e.g. 1 -> 1,2?
There was a problem hiding this comment.
In resharding as you described we would have:
0->0' ; 1->1', 2' ; 2->3' ; 3->4'
shard 0', 1', 3', 4' each will get all of the ougoing receipts from their respective parent shard
shard 2' will get no outgoing receipts
There was a problem hiding this comment.
Maybe I was wondering how this remapping happen for non split shards (e.g. shard 2 and 3 in the example above) since the function seem to care only about split shard ids.
There was a problem hiding this comment.
For non-split shards we treat it as splitting a parent shard into a single child shard. So it is covered here. In the example above shard 3' will get all of the outgoing receipts from its parent shard 2 and shard 4' will get all of the outgoing receipts from its parent 3.
Currently because of how storage keys work we need to copy all of the data from shards that are not split anyway. In the future we'll likely optimize that. Assuming we decide to have unique, incremental shard ids (e.g. 0, 1, 2, 3 -> (splitting 1) -> 0, 2, 3, 4, 5) then the outgoing receipts reassignment will be to keep the receipts where they are for non-split shards and reassign to lowest index child for split shards.
…ild (#9362) As described on [zulip](https://near.zulipchat.com/#narrow/stream/295558-pagoda.2Fcore/topic/outgoing.20receipts.20and.20resharding) the current way of reassigning outgoing receipts from parent shard to children shards won't work for any future reshardings. It only worked for 1->4 because there all children share the same parent. This PR implements a new approach where all outgoing receipts are going to be assigned to the child with the lowest shard id.
…ild (near#9362) As described on [zulip](https://near.zulipchat.com/#narrow/stream/295558-pagoda.2Fcore/topic/outgoing.20receipts.20and.20resharding) the current way of reassigning outgoing receipts from parent shard to children shards won't work for any future reshardings. It only worked for 1->4 because there all children share the same parent. This PR implements a new approach where all outgoing receipts are going to be assigned to the child with the lowest shard id.
As described on zulip the current way of reassigning outgoing receipts from parent shard to children shards won't work for any future reshardings. It only worked for 1->4 because there all children share the same parent.
This PR implements a new approach where all outgoing receipts are going to be assigned to the child with the lowest shard id.