Add support for block proposals to getblocktemplate method

[arya2]

Motivation

zcashd supports the proposal capability, so that if miners want to modify the template and then check that their modifications are still valid before mining, they could propose the new template.

This would also be useful for testing purposes, as we could propose the getblocktemplate's response and confirm that the block will be accepted if it has a valid solution.

This would require a Request enum in the block verifier with a variant for speculatively validating a block without checking its solution or committing it.

It also requires a new state request.

See #5630 for the state request.

Specifications

The block data MUST be validated and checked against the server's usual acceptance rules (excluding the check for a valid proof-of-work)

https://en.bitcoin.it/wiki/BIP_0023#Block_Proposal
https://en.bitcoin.it/wiki/BIP_0022#appendix-example-rejection-reasons

Designs

• Add a new block verifier request
  • Skip check::equihash_solution_is_valid(&block.header)
    (https://github.com/ZcashFoundation/zebra/blob/main/zebra-consensus/src/block.rs#L166)
  • Use CheckContextualValidity request from change(rpc): check contextual validity of getblocktemplate response #5630 instead of CommitBlock.
• If mode == GetBlockTemplateMode::Proposal
  • Deserialize data into the block
  • Call BlockVerifier with the new request variant instead of generating a template
  • Return null if successful or "rejected" if the block failed validation.

See zcashd impl: https://github.com/zcash/zcash/blob/master/src/rpc/mining.cpp#L538-L568

Out Of Scope:

• Detailed error handling

[teor2345]

All the miners we've seen so far use submitblock, so I think this RPC feature is optional.

[arya2]

All the miners we've seen so far use submitblock, so I think this RPC feature is optional.

It would be really convenient for testing, but this does seem like a potentially bad case of scope creep.

[teor2345]

All the miners we've seen so far use submitblock, so I think this RPC feature is optional.

It would be really convenient for testing, but this does seem like a potentially bad case of scope creep.

Let's think about how we can minimise the scope?

• don't return detailed errors
• split the verification function into PoW (equihash, header hash/difficulty) and the rest of the verification (difficulty target, header, transactions), we should be doing PoW first anyway for security
• add new chain verifier, block verifier, and state request types

Hmm that still seems like a lot.

[arya2]

Let's think about how we can minimise the scope?

I added a design, it's not as bad if we can skip returning detailed errors.

[mpguerra]

I really like this idea for testing! Let's definitely see if we can minimise the scope

[teor2345]

I really like this idea for testing! Let's definitely see if we can minimise the scope

I like this idea too!

If we design it right, we can add a feature that automatically tests each getblocktemplate RPC response before we send it. (Convert the response to a block, then call the proposal function with that block, then assert that it is valid.)

We might be able to automate this test, as long as we ignore errors when the height has changed. (There's a race condition between creating and checking the block.)

We will get better test coverage if we do this with a populated mempool.

[mpguerra]

@arya2 can you please add an estimate here?