Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change(test): Create test harness for calling getblocktemplate in proposal mode, but don't use it yet #5884

Merged
merged 48 commits into from
Jan 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
37d5afd
adds ValidateBlock request to state
arya2 Dec 15, 2022
6eb6048
adds `Request` enum in block verifier
arya2 Dec 15, 2022
162dedb
uses new Request in references to chain verifier
arya2 Dec 15, 2022
f7c5941
adds getblocktemplate proposal mode response type
arya2 Dec 15, 2022
9e0b8d1
makes getblocktemplate-rpcs feature in zebra-consensus select getbloc…
arya2 Dec 15, 2022
8420b75
Adds PR review revisions
arya2 Dec 16, 2022
e3b6d28
adds info log in CheckBlockProposalValidity
arya2 Dec 16, 2022
95cea02
Reverts replacement of match statement
arya2 Dec 16, 2022
31472a8
adds `GetBlockTemplate::capabilities` fn
arya2 Dec 16, 2022
ba302f7
conditions calling checkpoint verifier on !request.is_proposal
arya2 Dec 16, 2022
02b6758
updates references to validate_and_commit_non_finalized
arya2 Dec 16, 2022
2faa823
adds snapshot test, updates test vectors
arya2 Dec 16, 2022
32e6ac9
adds `should_count_metrics` to NonFinalizedState
arya2 Dec 16, 2022
a2ae3fb
Returns an error from chain verifier for block proposal requests belo…
arya2 Dec 16, 2022
40d002a
adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD
arya2 Dec 17, 2022
1b658a1
Merge branch 'main' into gbt-proposal-mode
arya2 Dec 19, 2022
da1b308
adds back block::Request to zebra-consensus lib
arya2 Dec 19, 2022
cb26b8a
updates snapshots
arya2 Dec 19, 2022
78e95d9
Removes unnecessary network arg
arya2 Dec 22, 2022
0c55a67
skips req in tracing intstrument for read state
arya2 Dec 22, 2022
616d280
Moves out block proposal validation to its own fn
arya2 Dec 22, 2022
8978541
corrects `difficulty_threshold_is_valid` docs
arya2 Dec 23, 2022
4551cf0
Update zebra-state/src/service.rs
arya2 Jan 5, 2023
e888039
Merge branch 'main' into gbt-proposal-mode
arya2 Jan 5, 2023
5cc7378
Apply suggestions from code review
arya2 Jan 10, 2023
67bf095
Update zebra-rpc/src/methods/get_block_template_rpcs.rs
arya2 Jan 10, 2023
f774822
check best chain tip
arya2 Jan 9, 2023
d75e17a
Update zebra-state/src/service.rs
arya2 Jan 10, 2023
9ce7d8a
Applies cleanup suggestions from code review
arya2 Jan 10, 2023
8b89cba
updates gbt acceptance test to make a block proposal
arya2 Dec 19, 2022
0bf81c7
fixes json parsing mistake
arya2 Dec 19, 2022
1d043b1
adds retries
arya2 Dec 22, 2022
da54e85
returns reject reason if there are no retries left
arya2 Dec 22, 2022
ee0fc0b
moves result deserialization to RPCRequestClient method, adds docs, m…
arya2 Jan 2, 2023
a6ac270
moves sleep(EXPECTED_TX_TIME) out of loop
arya2 Jan 2, 2023
d58d474
updates/adds info logs in retry loop
arya2 Jan 2, 2023
f105b58
Revert "moves sleep(EXPECTED_TX_TIME) out of loop"
arya2 Jan 2, 2023
fc4bd13
adds `allow(dead_code)`
arya2 Jan 3, 2023
509f890
Merge branch 'main' into gbt-proposal-test
mergify[bot] Jan 12, 2023
516f17c
tests with curtime, mintime, & maxtime
arya2 Jan 12, 2023
60c7a13
Fixes doc comment
arya2 Jan 12, 2023
d2cae99
Logs error responses from chain_verifier CheckProposal requests
arya2 Jan 13, 2023
bb32c52
Removes retry loop, adds num_txs log
arya2 Jan 13, 2023
71c335f
removes verbose info log
arya2 Jan 13, 2023
f052448
sorts mempool_txs before generating merkle root
arya2 Jan 13, 2023
dc3f2fc
Merge branch 'fix-gbt-merkle-root' into gbt-proposal-test
arya2 Jan 13, 2023
8c51e13
Make imports conditional on a feature
arya2 Jan 16, 2023
dd2438b
Disable new CI tests until bugs are fixed
arya2 Jan 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions zebra-chain/src/work/equihash.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ impl Solution {

Ok(())
}

#[cfg(feature = "getblocktemplate-rpcs")]
/// Returns a [`Solution`] of `[0; SOLUTION_SIZE]` to be used in block proposals.
pub fn for_proposal() -> Self {
Self([0; SOLUTION_SIZE])
}
}

impl PartialEq<Solution> for Solution {
Expand Down
2 changes: 1 addition & 1 deletion zebra-rpc/src/methods.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1193,7 +1193,7 @@ pub enum GetBlock {
///
/// Also see the notes for the [`Rpc::get_best_block_hash`] and `get_block_hash` methods.
#[derive(Copy, Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct GetBlockHash(#[serde(with = "hex")] block::Hash);
pub struct GetBlockHash(#[serde(with = "hex")] pub block::Hash);

/// Response to a `z_gettreestate` RPC request.
///
Expand Down
18 changes: 3 additions & 15 deletions zebra-rpc/src/methods/get_block_template_rpcs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ use crate::methods::{
},
get_block_template::{
check_miner_address, check_synced_to_tip, fetch_mempool_transactions,
fetch_state_tip_and_local_time, generate_coinbase_and_roots, validate_block_proposal,
fetch_state_tip_and_local_time, validate_block_proposal,
},
types::{
get_block_template::GetBlockTemplate, get_mining_info, hex_data::HexData,
Expand Down Expand Up @@ -566,24 +566,12 @@ where

// - After this point, the template only depends on the previously fetched data.

// Generate the coinbase transaction and default roots
//
// TODO: move expensive root, hash, and tree cryptography to a rayon thread?
let (coinbase_txn, default_roots) = generate_coinbase_and_roots(
let response = GetBlockTemplate::new(
network,
next_block_height,
miner_address,
&mempool_txs,
chain_tip_and_local_time.history_tree.clone(),
COINBASE_LIKE_ZCASHD,
);

let response = GetBlockTemplate::new(
&chain_tip_and_local_time,
server_long_poll_id,
coinbase_txn,
&mempool_txs,
default_roots,
mempool_txs,
submit_old,
COINBASE_LIKE_ZCASHD,
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ where
+ Sync
+ 'static,
{
let Ok(block) = block_proposal_bytes.zcash_deserialize_into() else {
let Ok(block) = block_proposal_bytes.zcash_deserialize_into::<block::Block>() else {
return Ok(ProposalRejectReason::Rejected.into())
};

Expand All @@ -125,7 +125,14 @@ where

Ok(chain_verifier_response
.map(|_hash| ProposalResponse::Valid)
.unwrap_or_else(|_| ProposalRejectReason::Rejected.into())
.unwrap_or_else(|verify_chain_error| {
tracing::info!(
verify_chain_error,
"Got error response from chain_verifier CheckProposal request"
);

ProposalRejectReason::Rejected.into()
})
.into())
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
//! The `GetBlockTempate` type is the output of the `getblocktemplate` RPC method.
//! The `GetBlockTempate` type is the output of the `getblocktemplate` RPC method in the
//! default 'template' mode. See [`ProposalResponse`] for the output in 'proposal' mode.

use zebra_chain::{
amount,
block::{ChainHistoryBlockTxAuthCommitmentHash, MAX_BLOCK_BYTES, ZCASH_BLOCK_VERSION},
parameters::Network,
serialization::DateTime32,
transaction::VerifiedUnminedTx,
transparent,
work::difficulty::{CompactDifficulty, ExpandedDifficulty},
};
use zebra_consensus::MAX_BLOCK_SIGOPS;
Expand All @@ -16,6 +19,7 @@ use crate::methods::{
GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD, GET_BLOCK_TEMPLATE_MUTABLE_FIELD,
GET_BLOCK_TEMPLATE_NONCE_RANGE_FIELD,
},
get_block_template::generate_coinbase_and_roots,
types::{
default_roots::DefaultRoots, long_poll::LongPollId, transaction::TransactionTemplate,
},
Expand All @@ -24,8 +28,10 @@ use crate::methods::{
};

pub mod parameters;
pub mod proposal;

pub use parameters::*;
pub use proposal::*;

/// A serialized `getblocktemplate` RPC response in template mode.
#[derive(Clone, Debug, Eq, PartialEq, serde::Serialize, serde::Deserialize)]
Expand Down Expand Up @@ -177,11 +183,11 @@ impl GetBlockTemplate {
/// If `like_zcashd` is true, try to match the coinbase transactions generated by `zcashd`
/// in the `getblocktemplate` RPC.
pub fn new(
network: Network,
miner_address: transparent::Address,
chain_tip_and_local_time: &GetBlockTemplateChainInfo,
long_poll_id: LongPollId,
coinbase_txn: TransactionTemplate<amount::NegativeOrZero>,
mempool_txs: &[VerifiedUnminedTx],
default_roots: DefaultRoots,
mempool_txs: Vec<VerifiedUnminedTx>,
submit_old: Option<bool>,
like_zcashd: bool,
) -> Self {
Expand All @@ -190,20 +196,41 @@ impl GetBlockTemplate {
(chain_tip_and_local_time.tip_height + 1).expect("tip is far below Height::MAX");

// Convert transactions into TransactionTemplates
let mut mempool_txs: Vec<TransactionTemplate<amount::NonNegative>> =
mempool_txs.iter().map(Into::into).collect();
let mut mempool_txs_with_templates: Vec<(
TransactionTemplate<amount::NonNegative>,
VerifiedUnminedTx,
)> = mempool_txs
.into_iter()
.map(|tx| ((&tx).into(), tx))
.collect();

// Transaction selection returns transactions in an arbitrary order,
// but Zebra's snapshot tests expect the same order every time.
if like_zcashd {
// Sort in serialized data order, excluding the length byte.
// `zcashd` sometimes seems to do this, but other times the order is arbitrary.
mempool_txs.sort_by_key(|tx| tx.data.clone());
mempool_txs_with_templates.sort_by_key(|(tx_template, _tx)| tx_template.data.clone());
} else {
// Sort by hash, this is faster.
mempool_txs.sort_by_key(|tx| tx.hash.bytes_in_display_order());
mempool_txs_with_templates
.sort_by_key(|(tx_template, _tx)| tx_template.hash.bytes_in_display_order());
}

let (mempool_tx_templates, mempool_txs): (Vec<_>, Vec<_>) =
mempool_txs_with_templates.into_iter().unzip();

// Generate the coinbase transaction and default roots
//
// TODO: move expensive root, hash, and tree cryptography to a rayon thread?
let (coinbase_txn, default_roots) = generate_coinbase_and_roots(
network,
next_block_height,
miner_address,
&mempool_txs,
chain_tip_and_local_time.history_tree.clone(),
like_zcashd,
);

// Convert difficulty
let target = chain_tip_and_local_time
.expected_difficulty
Expand All @@ -228,7 +255,7 @@ impl GetBlockTemplate {
final_sapling_root_hash: default_roots.block_commitments_hash,
default_roots,

transactions: mempool_txs,
transactions: mempool_tx_templates,

coinbase_txn,

Expand Down Expand Up @@ -259,33 +286,6 @@ impl GetBlockTemplate {
}
}

/// Error response to a `getblocktemplate` RPC request in proposal mode.
#[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum ProposalRejectReason {
/// Block proposal rejected as invalid.
Rejected,
}
arya2 marked this conversation as resolved.
Show resolved Hide resolved

/// Response to a `getblocktemplate` RPC request in proposal mode.
///
/// See <https://en.bitcoin.it/wiki/BIP_0023#Block_Proposal>
#[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(untagged, rename_all = "kebab-case")]
pub enum ProposalResponse {
/// Block proposal was rejected as invalid, returns `reject-reason` and server `capabilities`.
ErrorResponse {
/// Reason the proposal was invalid as-is.
reject_reason: ProposalRejectReason,

/// The getblocktemplate RPC capabilities supported by Zebra.
capabilities: Vec<String>,
},

/// Block proposal was successfully validated, returns null.
Valid,
}

#[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(untagged)]
/// A `getblocktemplate` RPC response.
Expand All @@ -296,30 +296,3 @@ pub enum Response {
/// `getblocktemplate` RPC request in proposal mode.
ProposalMode(ProposalResponse),
}

impl From<ProposalRejectReason> for ProposalResponse {
fn from(reject_reason: ProposalRejectReason) -> Self {
Self::ErrorResponse {
reject_reason,
capabilities: GetBlockTemplate::capabilities(),
}
}
}

impl From<ProposalRejectReason> for Response {
fn from(error_response: ProposalRejectReason) -> Self {
Self::ProposalMode(ProposalResponse::from(error_response))
}
}

impl From<ProposalResponse> for Response {
fn from(proposal_response: ProposalResponse) -> Self {
Self::ProposalMode(proposal_response)
}
}

impl From<GetBlockTemplate> for Response {
fn from(template: GetBlockTemplate) -> Self {
Self::TemplateMode(Box::new(template))
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
//! `ProposalResponse` is the output of the `getblocktemplate` RPC method in 'proposal' mode.
use super::{GetBlockTemplate, Response};

/// Error response to a `getblocktemplate` RPC request in proposal mode.
///
/// See <https://en.bitcoin.it/wiki/BIP_0022#Appendix:_Example_Rejection_Reasons>
#[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum ProposalRejectReason {
/// Block proposal rejected as invalid.
Rejected,
}

/// Response to a `getblocktemplate` RPC request in proposal mode.
///
/// See <https://en.bitcoin.it/wiki/BIP_0023#Block_Proposal>
#[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(untagged, rename_all = "kebab-case")]
pub enum ProposalResponse {
/// Block proposal was rejected as invalid, returns `reject-reason` and server `capabilities`.
ErrorResponse {
/// Reason the proposal was invalid as-is.
reject_reason: ProposalRejectReason,

/// The getblocktemplate RPC capabilities supported by Zebra.
capabilities: Vec<String>,
},

/// Block proposal was successfully validated, returns null.
Valid,
}

impl From<ProposalRejectReason> for ProposalResponse {
fn from(reject_reason: ProposalRejectReason) -> Self {
Self::ErrorResponse {
reject_reason,
capabilities: GetBlockTemplate::capabilities(),
}
}
}

impl From<ProposalRejectReason> for Response {
fn from(error_response: ProposalRejectReason) -> Self {
Self::ProposalMode(ProposalResponse::from(error_response))
}
}

impl From<ProposalResponse> for Response {
fn from(proposal_response: ProposalResponse) -> Self {
Self::ProposalMode(proposal_response)
}
}

impl From<GetBlockTemplate> for Response {
fn from(template: GetBlockTemplate) -> Self {
Self::TemplateMode(Box::new(template))
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ where
{
/// The hex-encoded serialized data for this transaction.
#[serde(with = "hex")]
pub(crate) data: SerializedTransaction,
pub data: SerializedTransaction,

/// The transaction ID of this transaction.
#[serde(with = "hex")]
Expand Down
1 change: 1 addition & 0 deletions zebrad/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ tonic-build = { version = "0.8.0", optional = true }
[dev-dependencies]
abscissa_core = { version = "0.5", features = ["testing"] }
hex = "0.4.3"
jsonrpc-core = "18.0.0"
once_cell = "1.17.0"
regex = "1.7.1"
semver = "1.0.16"
Expand Down
Loading