docs: documentation updates to abci (#18625)

UPGRADING.md

@@ -192,7 +192,7 @@ is `BeginBlock` -> `DeliverTx` (for all txs) -> `EndBlock`.
 ABCI++ 2.0 also brings `ExtendVote` and `VerifyVoteExtension` ABCI methods. These
 methods allow applications to extend and verify pre-commit votes. The Cosmos SDK
 allows an application to define handlers for these methods via `ExtendVoteHandler`
-and `VerifyVoteExtensionHandler` respectively. Please see [here](https://docs.cosmos.network/v0.50/building-apps/vote-extensions)
+and `VerifyVoteExtensionHandler` respectively. Please see [here](https://docs.cosmos.network/v0.50/build/abci/03-vote-extensions)
 for more info.
 
 #### Set PreBlocker

baseapp/abci.go

@@ -641,6 +641,8 @@ func (app *BaseApp) ExtendVote(_ context.Context, req *abci.RequestExtendVote) (
 // logic in verifying a vote extension from another validator during the pre-commit
 // phase. The response MUST be deterministic. An error is returned if vote
 // extensions are not enabled or if verifyVoteExt fails or panics.
+// We highly recommend a size validation due to performance degredation,
+// see more here https://docs.cometbft.com/v0.38/qa/cometbft-qa-38#vote-extensions-testbed
 func (app *BaseApp) VerifyVoteExtension(req *abci.RequestVerifyVoteExtension) (resp *abci.ResponseVerifyVoteExtension, err error) {
 	if app.verifyVoteExt == nil {
 		return nil, errors.New("application VerifyVoteExtension handler not set")

docs/build/abci/00-introduction.md

@@ -0,0 +1,51 @@
+# Introduction
+
+## What is ABCI?
+
+ABC, Application Blockchain Interface is the interface between CometBFT and the application, more information about ABCI can be found [here](https://docs.cometbft.com/v0.38/spec/abci/). Within the release of ABCI 2.0 for the 0.38 CometBFT release there were additional methods introduced.
+
+The 5 methods introduced during ABCI 2.0 are:
+
+* `PrepareProposal`
+* `ProcessProposal`
+* `ExtendVote`
+* `VerifyVoteExtension`
+* `FinalizeBlock`
+
+
+## The Flow
+
+## PrepareProposal
+
+Based on their voting power, CometBFT chooses a block proposer and calls `PrepareProposal` on the block proposer's application (Cosmos SDK). The selected block proposer is responsible for collecting outstanding transactions from the mempool, adhering to the application's specifications. The application can enforce custom transaction ordering and incorporate additional transactions, potentially generated from vote extensions in the previous block.
+
+To perform this manipulation on the application side, a custom handler must be implemented. By default, the Cosmos SDK provides `PrepareProposalHandler`, used in conjunction with an application specific mempool. A custom handler can be written by application developer, if a noop handler provided, all transactions are considered valid. Please see [this](https://github.com/fatal-fruit/abci-workshop) tutorial for more information on custom handlers.
+
+Please note that vote extensions will only be available on the following height in which vote extensions are enabled. More information about vote extensions can be found [here](https://docs.cosmos.network/main/build/abci/03-vote-extensions.md).
+
+After creating the proposal, the proposer returns it to CometBFT.
+
+PrepareProposal CAN be non-deterministic.
+
+## ProcessProposal
+
+This method allows validators to perform application-specific checks on the block proposal and is called on all validators. This is an important step in the consensus process, as it ensures that the block is valid and meets the requirements of the application. For example, validators could check that the block contains all the required transactions or that the block does not create any invalid state transitions.
+
+The implementation of `ProcessProposal` MUST be deterministic.
+
+## ExtendVote and VerifyVoteExtensions
+
+These methods allow applications to extend the voting process by requiring validators to perform additional actions beyond simply validating blocks.
+
+If vote extensions are enabled, `ExtendVote` will be called on every validator and each one will return its vote extension which is in practice a bunch of bytes. As mentioned above this data (vote extension) can only be retrieved in the next block height during `PrepareProposal`. Additionally, this data can be arbitrary, but in the provided tutorials, it serves as an oracle or proof of transactions in the mempool. Essentially, vote extensions are processed and injected as transactions. Examples of use-cases for vote extensions include prices for a price oracle or encryption shares for an encrypted transaction mempool. `ExtendVote` CAN be non-deterministic.
+
+`VerifyVoteExtensions` is performed on every validator multiple times in order to verify other validators' vote extensions. This check is submitted to validate the integrity and validity of the vote extensions preventing malicious or invalid vote extensions.
+
+Additionally, applications must keep the vote extension data concise as it can degrade the performance of their chain, see testing results [here](https://docs.cometbft.com/v0.38/qa/cometbft-qa-38#vote-extensions-testbed).
+
+`VerifyVoteExtensions` MUST be deterministic.
+
+
+## FinalizeBlock
+
+`FinalizeBlock` is then called and is responsible for updating the state of the blockchain and making the block available to users

docs/build/abci/01-prepare-proposal.md

@@ -0,0 +1,45 @@
+## Prepare Proposal
+
+`PrepareProposal` handles construction of the block, meaning that when a proposer
+is preparing to propose a block, it requests the application to evaluate a
+`RequestPrepareProposal`, which contains a series of transactions from CometBFT's
+mempool. At this point, the application has complete control over the proposal.
+It can modify, delete, and inject transactions from its own app-side mempool into
+the proposal or even ignore all the transactions altogether. What the application
+does with the transactions provided to it by `RequestPrepareProposal` has no
+effect on CometBFT's mempool.
+
+Note, that the application defines the semantics of the `PrepareProposal` and it
+MAY be non-deterministic and is only executed by the current block proposer.
+
+Now, reading mempool twice in the previous sentence is confusing, lets break it down.
+CometBFT has a mempool that handles gossiping transactions to other nodes
+in the network. The order of these transactions is determined by CometBFT's mempool,
+using FIFO as the sole ordering mechanism. It's worth noting that the priority mempool
+in Comet was removed or deprecated.
+However, since the application is able to fully inspect
+all transactions, it can provide greater control over transaction ordering.
+Allowing the application to handle ordering enables the application to define how
+it would like the block constructed.
+
+The Cosmos SDK defines the `DefaultProposalHandler` type, which provides applications with
+`PrepareProposal` and `ProcessProposal` handlers. If you decide to implement your
+own `PrepareProposal` handler, you must be sure to ensure that the transactions
+selected DO NOT exceed the maximum block gas (if set) and the maximum bytes provided
+by `req.MaxBytes`.
+
+```go reference
+https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go
+```
+
+This default implementation can be overridden by the application developer in
+favor of a custom implementation in [`app.go`](./01-app-go-v2.md):
+
+```go
+prepareOpt := func(app *baseapp.BaseApp) {
+abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app)
+app.SetPrepareProposal(abciPropHandler.PrepareProposalHandler())
+}
+
+baseAppOptions = append(baseAppOptions, prepareOpt)
+```

docs/build/abci/02-process-proposal.md

@@ -0,0 +1,32 @@
+## Process Proposal
+
+`ProcessProposal` handles the validation of a proposal from `PrepareProposal`,
+which also includes a block header. Meaning, that after a block has been proposed
+the other validators have the right to vote on a block. The validator in the
+default implementation of `PrepareProposal` runs basic validity checks on each
+transaction.
+
+Note, `ProcessProposal` MAY NOT be non-deterministic, i.e. it must be deterministic.
+This means if `ProcessProposal` panics or fails and we reject, all honest validator
+processes will prevote nil and the CometBFT round will proceed again until a valid
+proposal is proposed.
+
+Here is the implementation of the default implementation:
+
+```go reference
+https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go#L153-L159
+```
+
+Like `PrepareProposal` this implementation is the default and can be modified by
+the application developer in [`app.go`](./01-app-go-v2.md). If you decide to implement
+your own `ProcessProposal` handler, you must be sure to ensure that the transactions
+provided in the proposal DO NOT exceed the maximum block gas and `maxtxbytes` (if set).
+
+```go
+processOpt := func(app *baseapp.BaseApp) {
+abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app)
+app.SetProcessProposal(abciPropHandler.ProcessProposalHandler())
+}
+
+baseAppOptions = append(baseAppOptions, processOpt)
+```

docs/build/building-apps/04-vote-extensions.md → docs/build/abci/03-vote-extensions.md

@@ -33,6 +33,9 @@ to consider the size of the vote extensions as they could increase latency in bl
 production. See [here](https://github.com/cometbft/cometbft/blob/v0.38.0-rc1/docs/qa/CometBFT-QA-38.md#vote-extensions-testbed)
 for more details.
 
+Click [here](https://docs.cosmos.network/main/user/tutorials/vote-extensions) if you would like a walkthrough of how to implement vote extensions.
+
+
 ## Verify Vote Extension
 
 Similar to extending a vote, an application can also verify vote extensions from
@@ -51,6 +54,9 @@ validators will share the same view of what vote extensions they verify dependin
 on how votes are propagated. See [here](https://github.com/cometbft/cometbft/blob/v0.38.0-rc1/spec/abci/abci++_methods.md#verifyvoteextension)
 for more details.
 
+Additionally, please keep in mind that performance can be degraded if vote extensions are too big (https://docs.cometbft.com/v0.38/qa/cometbft-qa-38#vote-extensions-testbed), so we highly recommend a size validation in `VerifyVoteExtensions`.
+
+
 ## Vote Extension Propagation
 
 The agreed upon vote extensions at height `H` are provided to the proposing validator

docs/build/building-apps/02-app-mempool.md

@@ -11,94 +11,18 @@ This sections describes how the app-side mempool can be used and replaced.
 Since `v0.47` the application has its own mempool to allow much more granular
 block building than previous versions. This change was enabled by
 [ABCI 1.0](https://github.com/cometbft/cometbft/blob/v0.37.0/spec/abci).
-Notably it introduces the `PrepareProposal` and `ProcessProposal` steps of ABCI++.
+Notably it introduces the `PrepareProposal` and `ProcessProposal` steps of ABCI++. For more information please see [here](../abci/00-introduction.md)
 
 :::note Pre-requisite Readings
 
 * [BaseApp](../../learn/advanced/00-baseapp.md)
+* [Abci](../abci/00-introduction.md)
 
 :::
 
-## Prepare Proposal
-
-`PrepareProposal` handles construction of the block, meaning that when a proposer
-is preparing to propose a block, it requests the application to evaluate a
-`RequestPrepareProposal`, which contains a series of transactions from CometBFT's
-mempool. At this point, the application has complete control over the proposal.
-It can modify, delete, and inject transactions from it's own app-side mempool into
-the proposal or even ignore all the transactions altogether. What the application
-does with the transactions provided to it by `RequestPrepareProposal` have no
-effect on CometBFT's mempool.
-
-Note, that the application defines the semantics of the `PrepareProposal` and it
-MAY be non-deterministic and is only executed by the current block proposer.
-
-Now, reading mempool twice in the previous sentence is confusing, lets break it down.
-CometBFT has a mempool that handles gossiping transactions to other nodes
-in the network. How these transactions are ordered is determined by CometBFT's
-mempool, typically FIFO. However, since the application is able to fully inspect
-all transactions, it can provide greater control over transaction ordering.
-Allowing the application to handle ordering enables the application to define how
-it would like the block constructed. 
-
-The Cosmos SDK defines the `DefaultProposalHandler` type, which provides applications with
-`PrepareProposal` and `ProcessProposal` handlers. If you decide to implement your
-own `PrepareProposal` handler, you must be sure to ensure that the transactions
-selected DO NOT exceed the maximum block gas (if set) and the maximum bytes provided
-by `req.MaxBytes`.
-
-```go reference
-https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go
-```
-
-This default implementation can be overridden by the application developer in
-favor of a custom implementation in [`app.go`](./01-app-go-v2.md):
-
-```go
-prepareOpt := func(app *baseapp.BaseApp) {
-	abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app)
-	app.SetPrepareProposal(abciPropHandler.PrepareProposalHandler())
-}
-
-baseAppOptions = append(baseAppOptions, prepareOpt)
-```
-
-## Process Proposal
-
-`ProcessProposal` handles the validation of a proposal from `PrepareProposal`,
-which also includes a block header. Meaning, that after a block has been proposed
-the other validators have the right to vote on a block. The validator in the
-default implementation of `PrepareProposal` runs basic validity checks on each
-transaction.
-
-Note, `ProcessProposal` MAY NOT be non-deterministic, i.e. it must be deterministic.
-This means if `ProcessProposal` panics or fails and we reject, all honest validator
-processes will prevote nil and the CometBFT round will proceed again until a valid
-proposal is proposed.
-
-Here is the implementation of the default implementation:
-
-```go reference
-https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go#L153-L159
-```
-
-Like `PrepareProposal` this implementation is the default and can be modified by
-the application developer in [`app.go`](./01-app-go-v2.md). If you decide to implement
-your own `ProcessProposal` handler, you must be sure to ensure that the transactions
-provided in the proposal DO NOT exceed the maximum block gas (if set).
-
-```go
-processOpt := func(app *baseapp.BaseApp) {
-	abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app)
-	app.SetProcessProposal(abciPropHandler.ProcessProposalHandler())
-}
-
-baseAppOptions = append(baseAppOptions, processOpt)
-```
-
 ## Mempool
 
-Now that we have walked through the `PrepareProposal` & `ProcessProposal`, we can move on to walking through the mempool. 
++ Before we delve into `PrepareProposal` and `ProcessProposal`, let's first walk through the mempool concepts.
 
 There are countless designs that an application developer can write for a mempool, the SDK opted to provide only simple mempool implementations.
 Namely, the SDK provides the following mempools: