[CHIA-1563] add function to run a *trusted* block and return additions and removals.

[arvidn]

This adds a function called additions_and_removals(), which is similar to run_block_generator() and run_block_generator2() but with some important differences.

additions_and_removals():

• assumes that the block is valid. It's only meant to be run on blocks we've already validated (and perhaps pulled out of the blockchain database).
• only return the created coins ("additions") and spent coins ("removals"). The created coins also include the optional hint.

motivation

The two main reasons to introduce this function are:

• It's faster than run_block_generator()
• It completes the two use cases chia-blockchain has for running blocks. Letting run_block_generator() be more specialized to always fully validate blocks (e.g. the signature).

running trusted blocks

In chia-blockchain we sometimes validate blocks completely (1). Ensuring their conditions hold, their signatures are correct, the proof of space is correct etc. This is done as part of syncing a node as well as keeping up with the chain and maintaining the mempool. When we receive a transaction for the mempool, we fully validate it before accepting it. When we receive an UnfinishedBlock, we fully validated it.

The other use case for running block generators is to just get additions and removals (2) to respond to an RPC or to rebuild the coin store during a reorg (or when adding an orphaned block). This, case 2, could be replaced by the cheaper call to `additions_and_removals().

The non-test call sites of get_name_puzzle_conditions() are:

• CostLogger.add_cost() - part of the spend-sim, to compute the cost of a block in the simulator. This requires the full `run_block_generator() to compute the cost. Case (1).
• Blockchain.run_single_block() - this is to catch up a ForkInfo object with all the additions and removals of a fork of the chain. We need to do this when adding orphaned blocks, or reorging. Case (2).
• batch_pre_validate_blocks() - this is run when validating full blocks, either because we're syncing or just received a new block extending the blockchain. Case (1).
• _run_generator() - this is called when validating an UnfinishedBlock. Case (1).
  FullNodeAPI.request_block_header() - this is an RPC called on existing, validated, blocks in the chain. Case (2).
• FullBlockApi.request_block_header() - To construct a block header from a full block, we need the additions and removals, so we recompute them from a trusted block. Case (2).

git grep output, for non-tests and non-imports:

git grep -B 1 -A 1 get_name_puzzle_conditions

chia/clvm/spend_sim.py-        program: BlockGenerator = simple_solution_generator(spend_bundle)
chia/clvm/spend_sim.py:        npc_result: NPCResult = get_name_puzzle_conditions(
chia/clvm/spend_sim.py-            program,
--
chia/consensus/block_creation.py-def compute_block_cost(generator: BlockGenerator, constants: ConsensusConstants, height: uint32) -> uint64:
chia/consensus/block_creation.py:    result: NPCResult = get_name_puzzle_conditions(
chia/consensus/block_creation.py-        generator, constants.MAX_BLOCK_COST_CLVM, mempool_mode=True, height=height, constants=constants
--
chia/consensus/blockchain.py-            assert block.foliage_transaction_block is not None
chia/consensus/blockchain.py:            npc = get_name_puzzle_conditions(
chia/consensus/blockchain.py-                block_generator,
--
chia/consensus/multiprocess_validation.py-                assert block_generator.program == block.transactions_generator
chia/consensus/multiprocess_validation.py:                npc_result = get_name_puzzle_conditions(
chia/consensus/multiprocess_validation.py-                    block_generator,
--
chia/consensus/multiprocess_validation.py-        assert block_generator.program == unfinished_block.transactions_generator
chia/consensus/multiprocess_validation.py:        npc_result: NPCResult = get_name_puzzle_conditions(
chia/consensus/multiprocess_validation.py-            block_generator,
--
chia/full_node/full_node_api.py-                functools.partial(
chia/full_node/full_node_api.py:                    get_name_puzzle_conditions,
chia/full_node/full_node_api.py-                    block_generator,

Performance

One of the main reasons to justify this function, rather than just calling run_block_generator() is the time you can save. Here's the benchmark for block 4671894 on mainnet. The plot below compares run_block_generator() (slowest), run_block_generator2() (middle) and additions_and_removals() (fastest).

[coveralls-official]

Pull Request Test Coverage Report for Build 11263059066

Details

• 96 of 97 (98.97%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.1%) to 83.736%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
crates/chia-consensus/src/gen/additions_and_removals.rs	65	66	98.48%

Totals
Change from base Build 11262793680:	0.1%
Covered Lines:	12804
Relevant Lines:	15291

---

💛 - Coveralls

[matt-o-how]

I like this direction