Notification-based block pinning

[skunert]

This PR implements notification-based pinning for blocks, based on the discussion in #12475. Some inspiration was also taken from #12476.

Goal

While components in the node are listening to notifications, they should have the chance to get block data for notifications they receive. A block may be pruned before potential consumers have the chance to inspect them. The idea is to pin blocks before sending out notifications. When the last notification for a block is dropped, the block is automatically unpinned in the backend.

Another use case is the new RPC API, where we want to pin blocks in the backend until clients have signaled that unpinning is fine for them.

Implementation summary

• The client pins blocks before notifications are sent out
• Notifications have a private field, UnpinHandle, which sends a message on drop to a worker that does the unpinning operation
• For the state, we use the existing pin functionality in the state-db. It prevents pruning of displaced branch state as well as pruning of finalized states.
• For the block bodies and justifications, a new reference counting cache is introduced in BlockchainDb. Data is loaded into the cache lazily if a pinned block would be pruned. Reference counting is done independently of value loading.
• Pruning logic remains exactly the same, file database is unaware of this.

Closes #12475

polkadot companion: paritytech/polkadot#6582

[michalkucharczyk]

Just a thought - maybe pinned_blocks_cache: Arc<RwLock<PinnedBlocksCache<Block>>> should be moved to a one more wrapper struct, which would encapsulate all methods like: clear_pinning_cache, insert_justifications_if_pinned, insert_persisted_body_if_pinned , bump_ref. I don't feel these methods really belong to BlockchainDB domain.

This would allow to extract pinned cache from BlockchainDb and prevent from bloating up this implementation. And also provide better means for testing some functions on basic level.

Maybe an extra wrapper struct is not really needed and mutex could be moved to PinnedBlocksCache which could expose thread-safe API.

[bkchr]

Looks good! Mainly nitpicks.

[lexnv]

LGTM! Thanks for adding this! This enables the ecosystem to fully utilize the chainHead methods of the new RPC spec V2 🙏

[skunert]

Just a thought - maybe pinned_blocks_cache: Arc<RwLock<PinnedBlocksCache<Block>>> should be moved to a one more wrapper struct, which would encapsulate all methods like: clear_pinning_cache, insert_justifications_if_pinned, insert_persisted_body_if_pinned , bump_ref. I don't feel these methods really belong to BlockchainDB domain.

This would allow to extract pinned cache from BlockchainDb and prevent from bloating up this implementation. And also provide better means for testing some functions on basic level.

Maybe an extra wrapper struct is not really needed and mutex could be moved to PinnedBlocksCache which could expose thread-safe API.

I think to make this feasible we would need some larger refactoring. Currently most usage I have seen is something like backend.blockchain().body(hash), so BlockchainDb is accessed directly from outside the backend. So to have the cache in between, it lives in the BlockchainDb.
I we want to clean that up, maybe that could be a separate effort if you are ok with that.

[skunert]

bot merge

[michalkucharczyk]

I think to make this feasible we would need some larger refactoring. Currently most usage I have seen is something like backend.blockchain().body(hash), so BlockchainDb is accessed directly from outside the backend. So to have the cache in between, it lives in the BlockchainDb. I we want to clean that up, maybe that could be a separate effort if you are ok with that.

My point was about moving those clear_pinning_cache, insert_justifications_if_pinned, insert_persisted_body_if_pinned , bump_ref methods to different scope. Encapsulate them in the struct that is actually implementing this functionality (aka single reponsibility principle or sth :)).

Than this struct could be pulled into BlockchainDB and used like this:

struct X_X_X_Cache {
   pinned_blocks_cache: Arc<RwLock<PinnedBlocksCache<Block>>>
}

impl X_X_X_Cache {
        fn clear_pinning_cache() {}
	fn insert_justifications_if_pinned(&self, hash: Block::Hash, justification: Justification) {
		let mut cache = self.pinned_blocks_cache.write();
		if !cache.contains(hash) {
			return
		}

		let justifications = Justifications::from(justification);
		cache.insert_justifications(hash, Some(justifications));
	}
        ....
}

struct BlockchainDb {
    pinned_cache: X_X_X_Cache;
}

impl BlockchainDb {
	fn prune_blocks(..) { ...
          self.pinned_cache.insert_justifications_if_pinned()
        }
}

As already talked in private chat I am absolutely fine with not touching this or doing it later as refactoring.