On startup, check that the finalized state blocks match the checkpoint hashes

[teor2345]

Motivation

Currently, Zebra doesn't check if its checkpoints match the blocks in its cached state. This means that Zebra can follow a chain fork for a very long time, even after restarts.

This has become more of an issue recently, because Zebra will soon be able to:

• mine its own blocks, and
• generate its own checkpoints.

Specifications

zcashd checks for a few well-known block hashes on startup, which are typically network upgrade activations.

Complex Code or Requirements

To avoid startup delays, Zebra can do these checks concurrently with the rest of its startup and sync tasks. This can happen in a similar way to the Zcash parameter downloads. (But it can be async, because the state service is async.)

zebra/zebra-consensus/src/chain.rs

Lines 242 to 251 in bc783a7

	let groth16_download_handle = spawn_blocking(move || {
	span.in_scope(|| {
	if !debug_skip_parameter_preload {
	// The lazy static initializer does the download, if needed,
	// and the file hash checks.
	lazy_static::initialize(&crate::groth16::GROTH16_PARAMETERS);
	tracing::info!("Groth16 pre-download and check task finished");
	}
	})
	});

Testing

We could manually insert a bad block hash into the database, then run this check and make sure it fails.

The success case will be covered by existing tests.

Related Work

This is part of a larger issue for the Zcash network - we don't have good monitoring for chain forks or other serious network events.

We also want to track the recent chain forks that an individual Zebra node has discovered in our metrics:

• Add metrics for chain fork work and lengths #5297

[mpguerra]

tentatively scheduling this for after the GBT work

[teor2345]

I had a bit of trouble deciding the priority for this ticket. I think it might be medium priority, because it is an easy fix for a consensus rule check we should be doing.

[mpguerra]

Hey team! Please add your planning poker estimate with Zenhub @arya2 @conradoplg @dconnolly @oxarbitrage @teor2345 @upbqdn