Skip to content

Commit

Permalink
feat: Adds ADR 038 - go plugin system (#404)
Browse files Browse the repository at this point in the history
* Revert "Merge pull request #270 from provenance-io/llama/add-fee-denom-change-proposal"

This reverts commit 7462a84, reversing
changes made to 23ba5dc.

* Bring in v0.46.4 changes. (#362)

* docs: add ApplicationQueryService release notes (cosmos#13587)

* docs: add ApplicationQueryService release notes

* updates

Co-authored-by: Julien Robert <[email protected]>

* feat(cli): add module-account cli cmd and grpc get api (backport cosmos#13612) (cosmos#13616)

* feat(cli): add module-account cli cmd and grpc get api (cosmos#13612)

(cherry picked from commit ddf5cf0)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/auth/v1beta1/query.pulsar.go
#	api/cosmos/auth/v1beta1/query_grpc.pb.go
#	x/auth/client/testutil/suite.go
#	x/auth/types/query.pb.go

* fix conflicts

* updates

* updates

Co-authored-by: Sai Kumar <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (backport cosmos#12474) (cosmos#13598)

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (cosmos#12474)

(cherry picked from commit 18da0e9)

# Conflicts:
#	CHANGELOG.md

* fix changelog

* changelog

* fix: flakey test

Co-authored-by: Tyler <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix: app-hash mismatch if upgrade migration commit is interrupted (backport cosmos#13530) (cosmos#13627)

* feat(cli): Add iavl-disable-fastnode cmd flag with proper description (cosmos#13656) (cosmos#13659)

(cherry picked from commit c833190)

Co-authored-by: William Chong <[email protected]>

* build(deps): Bump github.com/cosmos/iavl from 0.19.3 to 0.19.4 (cosmos#13680)

* feat: emit cached context events (backport cosmos#13063) (cosmos#13702)

* chore: prepare 0.46.4 changelog (cosmos#13716)

* chore: prepare 0.46.4 changelog

* wording

* updates

* updates

* Change the default for the iavl-disable-fastnode flag to true (from false) to match the actual config default.

* Add changelog entry for the iavl fastnode flag default.

* Update swagger docs.

Co-authored-by: Robert Zaremba <[email protected]>
Co-authored-by: Julien Robert <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Sai Kumar <[email protected]>
Co-authored-by: Tyler <[email protected]>
Co-authored-by: William Chong <[email protected]>

* Mark v0.46.4-pio-1 in the changelog and release notes.

* Bring in v0.46.5 changes. (#365)

* Bring in changes from Cosmos-SDK v0.46.6 (#367)

* docs: add ApplicationQueryService release notes (cosmos#13587)

* docs: add ApplicationQueryService release notes

* updates

Co-authored-by: Julien Robert <[email protected]>

* feat(cli): add module-account cli cmd and grpc get api (backport cosmos#13612) (cosmos#13616)

* feat(cli): add module-account cli cmd and grpc get api (cosmos#13612)

(cherry picked from commit ddf5cf0)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/auth/v1beta1/query.pulsar.go
#	api/cosmos/auth/v1beta1/query_grpc.pb.go
#	x/auth/client/testutil/suite.go
#	x/auth/types/query.pb.go

* fix conflicts

* updates

* updates

Co-authored-by: Sai Kumar <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (backport cosmos#12474) (cosmos#13598)

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (cosmos#12474)

(cherry picked from commit 18da0e9)

# Conflicts:
#	CHANGELOG.md

* fix changelog

* changelog

* fix: flakey test

Co-authored-by: Tyler <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix: app-hash mismatch if upgrade migration commit is interrupted (backport cosmos#13530) (cosmos#13627)

* feat(cli): Add iavl-disable-fastnode cmd flag with proper description (cosmos#13656) (cosmos#13659)

(cherry picked from commit c833190)

Co-authored-by: William Chong <[email protected]>

* build(deps): Bump github.com/cosmos/iavl from 0.19.3 to 0.19.4 (cosmos#13680)

* feat: emit cached context events (backport cosmos#13063) (cosmos#13702)

* chore: prepare 0.46.4 changelog (cosmos#13716)

* chore: prepare 0.46.4 changelog

* wording

* updates

* updates

* ci: notify for 0.46.x releases (cosmos#13719)

* feat: notify for 0.46.x releases

* updates changelog

* docs: fix algolia on 0.46 (cosmos#13730)

* ci: modifying docs in 0.46 should not redeploy docs (cosmos#13732)

* ci: modifying docs in 0.46 should not redeploy docs

* updates

* fix: propagate msg events correctly in x/gov (backport cosmos#13728) (cosmos#13748)

* refactor: add error log when iavl set failed (backport cosmos#13803) (cosmos#13804)

* refactor: add error log when iavl set failed (cosmos#13803)

* add error log when iavl set failed

Ref: cosmos#12012

* Update CHANGELOG.md

* play safe

(cherry picked from commit 22f3261)

# Conflicts:
#	CHANGELOG.md

* fix conflicts

Co-authored-by: yihuang <[email protected]>

* chore: bump tendermint to `0.34.23` (cosmos#13814)

* fix: propagate events in x/group through sdk.Results (backport cosmos#13808) (cosmos#13809)

* docs: update algolia index (cosmos#13823)

* fix: bank store migration (backport cosmos#13821) (cosmos#13829)

* fix: bank store migration (cosmos#13821)

(cherry picked from commit d314a12)

# Conflicts:
#	x/bank/migrations/v3/store.go
#	x/bank/migrations/v3/store_test.go

* updates

Co-authored-by: billy rennekamp <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* feat(types): set custom GasConfig on Context for GasKVStore (backport cosmos#13826) (cosmos#13833)

* feat(types): set custom GasConfig on Context for GasKVStore (cosmos#13826)

(cherry picked from commit f001b46)

# Conflicts:
#	CHANGELOG.md
#	types/context.go
#	types/context_test.go

* fix conflicts

* updates

Co-authored-by: Federico Kunze Küllmer <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* chore: prepare 0.46.5 release (cosmos#13816)

* fix: Allow underscores in EventRegex (backport cosmos#13861) (cosmos#13864)

* fix: Allow underscores in EventRegex (cosmos#13861)

(cherry picked from commit 14c582f)

# Conflicts:
#	CHANGELOG.md
#	tests/e2e/tx/service_test.go

* updates

* updates

Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix(group): add group members weight checks (backport cosmos#13869) (cosmos#13880)

* fix(group): add group members weight checks (cosmos#13869)

(cherry picked from commit 3423442)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/tx/v1beta1/service.pulsar.go
#	types/tx/service.pb.go
#	x/group/keeper/keeper.go

* fix conflicts

* updates

* updates

Co-authored-by: Julien Robert <[email protected]>

* fix(group)!: Fix group min execution period (backport cosmos#13876) (cosmos#13885)

* fix(group)!: Fix group min execution period (cosmos#13876)

* fix: don't check MinExecutionPeriod in `Allow`

* Check MinExecutionPeriod on doExecuteMsgs

* Fix TestExec

* Fix TestExec

* test exec pruned

* Fix submitproposal

* Add changelog

* typo

* revert some changes

* add minExecutionPeriod

* Add docs and specs

(cherry picked from commit 7661f62)

# Conflicts:
#	CHANGELOG.md
#	x/group/README.md
#	x/group/keeper/keeper_test.go

* fix docs

* fix other conflicts

* fix test

Co-authored-by: Amaury <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* feat(bank): Add helper for v0.46 denom migration (cosmos#13891)

* feat(bank): Add helper for v0.46 denom migration

* CL

* Clearer name

* Update x/bank/migrations/v046/store.go

Co-authored-by: Likhita Polavarapu <[email protected]>

* rename

Co-authored-by: Likhita Polavarapu <[email protected]>

* chore: impove 0.46.5 release notes (cosmos#13898)

* chore: impove 0.46.5 release notes

* update changelog

* refactor: State Streaming Docs + Explicit Config Support (backport cosmos#13894) (cosmos#13909)

* chore: remove typo (cosmos#13914) (cosmos#13917)

(cherry picked from commit 34dcca7)

Co-authored-by: Julien Robert <[email protected]>

* fix: correctly propagate msg errors in gov (backport cosmos#13918) (cosmos#13928)

* fix: correctly propagate msg errors in gov (cosmos#13918)

* fix: correctly propagate msg errors in gov

* chore: update changelog

* fix: correctly check proposal status in tests

* chore: delete unused var

(cherry picked from commit 5581f7f)

# Conflicts:
#	CHANGELOG.md
#	x/gov/abci_test.go

* fix conflicts

* updates

Co-authored-by: John Letey <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* feat(bank): enable 0.46.5 bank migration fix through migrator keeper (cosmos#13929)

* Update changelog for version bump.

* Bring in Cosmos-SDK PR 13651 that overhauls/cleans up the GetConfig function to not require manual addition of stuff in order for it to be read.

* Fix the pruning command to use the existing viper instance (that has everything loaded into it) instead of creating a new one.

Co-authored-by: Robert Zaremba <[email protected]>
Co-authored-by: Julien Robert <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Sai Kumar <[email protected]>
Co-authored-by: Tyler <[email protected]>
Co-authored-by: William Chong <[email protected]>
Co-authored-by: yihuang <[email protected]>
Co-authored-by: billy rennekamp <[email protected]>
Co-authored-by: Federico Kunze Küllmer <[email protected]>
Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Amaury <[email protected]>
Co-authored-by: Likhita Polavarapu <[email protected]>
Co-authored-by: John Letey <[email protected]>

* Backport to v0.46.x: feat: Add x/authz SendAuthorization AllowList cosmos#12648 (#368)

* feat: Add x/authz SendAuthorization AllowList (cosmos#12648)

Closes: cosmos#12609

---

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)

* Fix a unit test that for whatever reason didn't get updated in the merge.

Co-authored-by: likhita-809 <[email protected]>

* Mark v0.46.6-pio-1 in the changelog and release notes.

* use hashicorp go-plugin

* python example

* implement streaming to baseapp

* abci listener wiring

* update test files

* update test files

* kafka plugin python example

* update package name

* update proto and services

* tx and kvpair indexing

* tidy up

* add readme

* update readme

* add testing section

* listeners need to be async

* remove goroutine on blocking call

* terminate with os.Exit on stopNodeOnErr

* dumb pipe bridge

* fix: listen state writes on rootmulti store only

and refactoring file streamer to output single file for each block

update spec

sort storeKey once at first

move MemoryListener store package

* introduce memory listener and OnCommit listener

* remove WriteListener and use MemoryListener

* internalize listener init to cache layer

* determenistic PopStateCache, grpc service

* add Go file plugin

* multiple service support

* add \n

* add proto dir

* append to file

* support in-process listeners

* use setStreamingService

* enable plugins as managed clients

* fix after cherry pick

* add default to toml config

* format toml config

* update template

* update template

* update documentation

* fix test

* write request to single topics

* fix lint errors

* remove python examples, CI cannot verify

* fix gosec findings

* move plugin proto with sdk proto files, regen plugin code

* update readme

* update deps

remove unused dep

* Bring in v0.46.7 updates. (#402)

* docs: add ApplicationQueryService release notes (cosmos#13587)

* docs: add ApplicationQueryService release notes

* updates

Co-authored-by: Julien Robert <[email protected]>

* feat(cli): add module-account cli cmd and grpc get api (backport cosmos#13612) (cosmos#13616)

* feat(cli): add module-account cli cmd and grpc get api (cosmos#13612)

(cherry picked from commit ddf5cf0)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/auth/v1beta1/query.pulsar.go
#	api/cosmos/auth/v1beta1/query_grpc.pb.go
#	x/auth/client/testutil/suite.go
#	x/auth/types/query.pb.go

* fix conflicts

* updates

* updates

Co-authored-by: Sai Kumar <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (backport cosmos#12474) (cosmos#13598)

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (cosmos#12474)

(cherry picked from commit 18da0e9)

# Conflicts:
#	CHANGELOG.md

* fix changelog

* changelog

* fix: flakey test

Co-authored-by: Tyler <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix: app-hash mismatch if upgrade migration commit is interrupted (backport cosmos#13530) (cosmos#13627)

* feat(cli): Add iavl-disable-fastnode cmd flag with proper description (cosmos#13656) (cosmos#13659)

(cherry picked from commit c833190)

Co-authored-by: William Chong <[email protected]>

* build(deps): Bump github.com/cosmos/iavl from 0.19.3 to 0.19.4 (cosmos#13680)

* feat: emit cached context events (backport cosmos#13063) (cosmos#13702)

* chore: prepare 0.46.4 changelog (cosmos#13716)

* chore: prepare 0.46.4 changelog

* wording

* updates

* updates

* ci: notify for 0.46.x releases (cosmos#13719)

* feat: notify for 0.46.x releases

* updates changelog

* docs: fix algolia on 0.46 (cosmos#13730)

* ci: modifying docs in 0.46 should not redeploy docs (cosmos#13732)

* ci: modifying docs in 0.46 should not redeploy docs

* updates

* fix: propagate msg events correctly in x/gov (backport cosmos#13728) (cosmos#13748)

* refactor: add error log when iavl set failed (backport cosmos#13803) (cosmos#13804)

* refactor: add error log when iavl set failed (cosmos#13803)

* add error log when iavl set failed

Ref: cosmos#12012

* Update CHANGELOG.md

* play safe

(cherry picked from commit 22f3261)

# Conflicts:
#	CHANGELOG.md

* fix conflicts

Co-authored-by: yihuang <[email protected]>

* chore: bump tendermint to `0.34.23` (cosmos#13814)

* fix: propagate events in x/group through sdk.Results (backport cosmos#13808) (cosmos#13809)

* docs: update algolia index (cosmos#13823)

* fix: bank store migration (backport cosmos#13821) (cosmos#13829)

* fix: bank store migration (cosmos#13821)

(cherry picked from commit d314a12)

# Conflicts:
#	x/bank/migrations/v3/store.go
#	x/bank/migrations/v3/store_test.go

* updates

Co-authored-by: billy rennekamp <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* feat(types): set custom GasConfig on Context for GasKVStore (backport cosmos#13826) (cosmos#13833)

* feat(types): set custom GasConfig on Context for GasKVStore (cosmos#13826)

(cherry picked from commit f001b46)

# Conflicts:
#	CHANGELOG.md
#	types/context.go
#	types/context_test.go

* fix conflicts

* updates

Co-authored-by: Federico Kunze Küllmer <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* chore: prepare 0.46.5 release (cosmos#13816)

* fix: Allow underscores in EventRegex (backport cosmos#13861) (cosmos#13864)

* fix: Allow underscores in EventRegex (cosmos#13861)

(cherry picked from commit 14c582f)

# Conflicts:
#	CHANGELOG.md
#	tests/e2e/tx/service_test.go

* updates

* updates

Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix(group): add group members weight checks (backport cosmos#13869) (cosmos#13880)

* fix(group): add group members weight checks (cosmos#13869)

(cherry picked from commit 3423442)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/tx/v1beta1/service.pulsar.go
#	types/tx/service.pb.go
#	x/group/keeper/keeper.go

* fix conflicts

* updates

* updates

Co-authored-by: Julien Robert <[email protected]>

* fix(group)!: Fix group min execution period (backport cosmos#13876) (cosmos#13885)

* fix(group)!: Fix group min execution period (cosmos#13876)

* fix: don't check MinExecutionPeriod in `Allow`

* Check MinExecutionPeriod on doExecuteMsgs

* Fix TestExec

* Fix TestExec

* test exec pruned

* Fix submitproposal

* Add changelog

* typo

* revert some changes

* add minExecutionPeriod

* Add docs and specs

(cherry picked from commit 7661f62)

# Conflicts:
#	CHANGELOG.md
#	x/group/README.md
#	x/group/keeper/keeper_test.go

* fix docs

* fix other conflicts

* fix test

Co-authored-by: Amaury <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* feat(bank): Add helper for v0.46 denom migration (cosmos#13891)

* feat(bank): Add helper for v0.46 denom migration

* CL

* Clearer name

* Update x/bank/migrations/v046/store.go

Co-authored-by: Likhita Polavarapu <[email protected]>

* rename

Co-authored-by: Likhita Polavarapu <[email protected]>

* chore: impove 0.46.5 release notes (cosmos#13898)

* chore: impove 0.46.5 release notes

* update changelog

* refactor: State Streaming Docs + Explicit Config Support (backport cosmos#13894) (cosmos#13909)

* chore: remove typo (cosmos#13914) (cosmos#13917)

(cherry picked from commit 34dcca7)

Co-authored-by: Julien Robert <[email protected]>

* fix: correctly propagate msg errors in gov (backport cosmos#13918) (cosmos#13928)

* fix: correctly propagate msg errors in gov (cosmos#13918)

* fix: correctly propagate msg errors in gov

* chore: update changelog

* fix: correctly check proposal status in tests

* chore: delete unused var

(cherry picked from commit 5581f7f)

# Conflicts:
#	CHANGELOG.md
#	x/gov/abci_test.go

* fix conflicts

* updates

Co-authored-by: John Letey <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* feat(bank): enable 0.46.5 bank migration fix through migrator keeper (cosmos#13929)

* chore: bump tendermint to 0.34.24 (cosmos#13972)

* chore: add group and gov/v1 in swagger (backport cosmos#13984) (cosmos#13996)

* chore: add group and gov/v1 in swagger (cosmos#13984)

Co-authored-by: Marko <[email protected]>
(cherry picked from commit cfa224c)

# Conflicts:
#	client/docs/config.json
#	client/docs/swagger-ui/swagger.yaml

* updates

Co-authored-by: Julien Robert <[email protected]>

* fix(baseapp): fix snapshot interval bug (backport cosmos#14049) (cosmos#14053)

* refactor: use viper unmarshal in `config.GetConfig` function (backport cosmos#13651) (cosmos#14082)

* fix(group)!: Don't re-tally proposals after VP end (backport cosmos#14071) (cosmos#14091)

* fix(group)!: Don't re-tally proposals after VP end (cosmos#14071)

(cherry picked from commit 4ebe3aa)

# Conflicts:
#	CHANGELOG.md

* fix changelog and test

Co-authored-by: Amaury <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix: do not shadow clientCtx in start.go (backport cosmos#14086) (cosmos#14101)

* fix: do not shadow clientCtx in start.go (cosmos#14086)

(cherry picked from commit f96072d)

# Conflicts:
#	server/start.go

* updates

Co-authored-by: Julien Robert <[email protected]>

* chore: Test for Server Config Read cosmos#14125 (cosmos#14127)

(cherry picked from commit 9f46665)

Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* fix: remove duplicate ante events (backport cosmos#13983) (cosmos#14142)

* fix: remove duplicate ante events (cosmos#13983)

## Description

After enabling the post-handler on our chain, we realised something odd with our transaction events. The ordering was `[ANTE, MSGS, ANTE, POST]`. After doing some digging, we realised that the context used inside the post-handler still contains the events from the ante-handler.

This should be backported to v0.46.x and v0.47.x 😄
Happy to make it compatible with cosmos#13940 when that gets merged!

cc @mbreithecker

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)

(cherry picked from commit 25449b5)

# Conflicts:
#	CHANGELOG.md

* fix changelog

Co-authored-by: John Letey <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* feat: support alternative query multistore (backport cosmos#13529) (cosmos#14169)

* feat: support alternative query multistore (cosmos#13529)

* support customize query multistore

* Update CHANGELOG.md

* fix test

* Update baseapp/abci.go

Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* Update baseapp/baseapp.go

Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* Update baseapp/options.go

Co-authored-by: Aleksandr Bezobchuk <[email protected]>

* Apply suggestions from code review

Co-authored-by: Marko <[email protected]>
Co-authored-by: Aleksandr Bezobchuk <[email protected]>
(cherry picked from commit 15accd7)

# Conflicts:
#	CHANGELOG.md
#	baseapp/baseapp.go
#	baseapp/options.go

* fix conflicts

Co-authored-by: yihuang <[email protected]>

* fix: state listener observe writes at wrong time (backport cosmos#13516) (cosmos#14138)

* fix: state listener observe writes at wrong time (cosmos#13516)

* fix: state listener observe writes at wrong time

Closes: cosmos#13457

Currently state listener is notified when the cache store write, which happens in commit event only, which breaks the current design.
The solution (as discussed in the issue) is to listen state writes on rootmulti store only.

It also changes the file streamer to output single data file for the writes in the whole block, since we can't distinguish writes from different stage of abci events.

It adds new config items for file streamer:
- streamers.file.output-metadata
- streamers.file.stop-node-on-error
- streamers.file.fsync

* synchronous abci call, and format doc

* fix comment

* update file streamer readme and fix typos

* typo

* fix: state listener observe writes at wrong time

Closes: cosmos#13457

Currently state listener is notified when the cache store write, which happens in commit event only, which breaks the current design.
The solution (as discussed in the issue) is to listen state writes on rootmulti store only.

It also changes the file streamer to output single data file for the writes in the whole block, since we can't distinguish writes from different stage of abci events.

It adds new config items for file streamer:
- streamers.file.output-metadata
- streamers.file.stop-node-on-error
- streamers.file.fsync

synchronous abci call, and format doc

fix comment

update file streamer readme and fix typos

typo

* improve UX of file streamer, make it immediately usable after enabled

- set default value to write_dir.
- make write_dir based on home directory by default.
- auto-create the directory if not exists.

* get homePage from opts

Co-authored-by: Marko <[email protected]>
(cherry picked from commit 1f91ee2)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/base/store/v1beta1/listening.pulsar.go
#	baseapp/streaming.go
#	docs/architecture/adr-038-state-listening.md
#	server/config/toml.go
#	simapp/app_v2.go
#	store/cachemulti/store.go
#	store/iavl/store.go
#	store/mem/store.go
#	store/streaming/constructor.go
#	store/streaming/file/service.go
#	store/streaming/file/service_test.go
#	store/types/listening.pb.go
#	store/types/store.go

* `make proto-gen`, update changelog, delete uncessary files

* fix conflicts

* fix conflicts

* revert api breaking change

* fix build

* fix unit test

Co-authored-by: yihuang <[email protected]>
Co-authored-by: Julien Robert <[email protected]>
Co-authored-by: Marko <[email protected]>

* refactor: provide a helper for baseapp options (backport cosmos#14175) (cosmos#14178)

* refactor: provide a helper for baseapp options (cosmos#14175)

* provide a helper for baseapp options

* rename

* changelog entry

* fix spelling

(cherry picked from commit 1b6192f)

# Conflicts:
#	CHANGELOG.md
#	server/mock/app_test.go
#	simapp/simd/cmd/root.go

* conflicts

* lint

Co-authored-by: Marko <[email protected]>

* feat: add --grpc client option (backport cosmos#14051) (cosmos#14192)

Co-authored-by: Aaron Craelius <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* refactor: cleanup store/streaming/constructor.go (backport cosmos#14044) (cosmos#14236)

* chore: audit store/streaming/file/service.go (backport cosmos#14234) (cosmos#14241)

Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* fix: fix grpc flag conflict (backport cosmos#14244) (cosmos#14248)

* chore: prepare 0.46.7 release (cosmos#14103)

* fix(gov): Fix v3 votes migrations (backport cosmos#14214) (cosmos#14277)

Co-authored-by: Amaury <[email protected]>
Co-authored-by: Julien Robert <[email protected]>

* Add changelog entry for bringing in v0.46.7 changes.

* Fix the changelog. The merge duplicated several SDK sections.

* Fix the TestFileStreamingService.

Co-authored-by: Robert Zaremba <[email protected]>
Co-authored-by: Julien Robert <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Sai Kumar <[email protected]>
Co-authored-by: Tyler <[email protected]>
Co-authored-by: William Chong <[email protected]>
Co-authored-by: yihuang <[email protected]>
Co-authored-by: billy rennekamp <[email protected]>
Co-authored-by: Federico Kunze Küllmer <[email protected]>
Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Amaury <[email protected]>
Co-authored-by: Likhita Polavarapu <[email protected]>
Co-authored-by: John Letey <[email protected]>
Co-authored-by: John Letey <[email protected]>
Co-authored-by: Marko <[email protected]>
Co-authored-by: Aaron Craelius <[email protected]>

* Mark v0.46.7-pio-1 in the changelog and release notes.

* Note v0.46.6-pio-3 in the changelog.

* cherry-pick cosmos-sdk authz validatebasic fix (#429)

* cherry-pick cosmos-sdk authz validatebasic fix

* Add change log entry

Co-authored-by: bruce-wayne2 <[email protected]>

* set res for delivertx

* Revert "Revert "Merge pull request #270 from provenance-io/llama/add-fee-denom-change-proposal""

This reverts commit 648ae0c.

* Mark v0.46.8-pio-1 in the changelog and release notes.

* improvements from upstream feedback

* fix tests

* remove release notes

* add changelog entry

* changes based on feedback

* fix tests

* default to empty keys

* feadback fixes

---------

Co-authored-by: Daniel Wedul <[email protected]>
Co-authored-by: Robert Zaremba <[email protected]>
Co-authored-by: Julien Robert <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Sai Kumar <[email protected]>
Co-authored-by: Tyler <[email protected]>
Co-authored-by: William Chong <[email protected]>
Co-authored-by: yihuang <[email protected]>
Co-authored-by: billy rennekamp <[email protected]>
Co-authored-by: Federico Kunze Küllmer <[email protected]>
Co-authored-by: Aleksandr Bezobchuk <[email protected]>
Co-authored-by: Amaury <[email protected]>
Co-authored-by: Likhita Polavarapu <[email protected]>
Co-authored-by: John Letey <[email protected]>
Co-authored-by: John Letey <[email protected]>
Co-authored-by: Marko <[email protected]>
Co-authored-by: Aaron Craelius <[email protected]>
Co-authored-by: Carlton Hanna <[email protected]>
Co-authored-by: bruce-wayne2 <[email protected]>
  • Loading branch information
20 people authored Feb 1, 2023
1 parent 3abe36f commit 393db96
Show file tree
Hide file tree
Showing 57 changed files with 3,616 additions and 1,825 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ vagrant
# IDE
.idea
*.iml
*.ipr
*.iws
.dir-locals.el
.vscode

Expand Down
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,9 @@ Ref: https://keepachangelog.com/en/1.0.0/

## Unreleased

* nothing
### Features

* [#404](https://github.com/provenance-io/cosmos-sdk/pull/404) Add ADR-038 State Listening with Go Plugin System

---

Expand Down
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,12 @@ test-sim-nondeterminism:
@go test -mod=readonly $(SIMAPP) -run TestAppStateDeterminism -Enabled=true \
-NumBlocks=100 -BlockSize=200 -Commit=true -Period=0 -v -timeout 24h

test-sim-nondeterminism-streaming:
@echo "Running non-determinism-streaming test..."
@go test -mod=readonly $(SIMAPP) -run TestAppStateDeterminism -Enabled=true \
-NumBlocks=100 -BlockSize=200 -Commit=true -Period=0 -v -timeout 24h \
-EnableStreaming=true

test-sim-custom-genesis-fast:
@echo "Running custom genesis simulation..."
@echo "By default, ${HOME}/.gaiad/config/genesis.json will be used."
Expand Down
61 changes: 42 additions & 19 deletions baseapp/abci.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@ import (
"time"

"github.com/gogo/protobuf/proto"
abci "github.com/tendermint/tendermint/abci/types"
tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
"github.com/hashicorp/go-plugin"
"google.golang.org/grpc/codes"
grpcstatus "google.golang.org/grpc/status"

abci "github.com/tendermint/tendermint/abci/types"
tmproto "github.com/tendermint/tendermint/proto/tendermint/types"

"github.com/cosmos/cosmos-sdk/codec"
snapshottypes "github.com/cosmos/cosmos-sdk/snapshots/types"
"github.com/cosmos/cosmos-sdk/telemetry"
Expand Down Expand Up @@ -186,10 +188,12 @@ func (app *BaseApp) BeginBlock(req abci.RequestBeginBlock) (res abci.ResponseBeg
// set the signed validators for addition to context in deliverTx
app.voteInfos = req.LastCommitInfo.GetVotes()

// call the hooks with the BeginBlock messages
for _, streamingListener := range app.abciListeners {
if err := streamingListener.ListenBeginBlock(app.deliverState.ctx, req, res); err != nil {
panic(fmt.Errorf("BeginBlock listening hook failed, height: %d, err: %w", req.Header.Height, err))
// call the streaming service hook with the BeginBlock messages
for _, abciListener := range app.streamingManager.AbciListeners {
ctx := app.deliverState.ctx
blockHeight := ctx.BlockHeight()
if err := abciListener.ListenBeginBlock(ctx, req, res); err != nil {
app.logger.Error("BeginBlock listening hook failed", "height", blockHeight, "err", err)
}
}

Expand All @@ -213,10 +217,12 @@ func (app *BaseApp) EndBlock(req abci.RequestEndBlock) (res abci.ResponseEndBloc
res.ConsensusParamUpdates = cp
}

// call the streaming service hooks with the EndBlock messages
for _, streamingListener := range app.abciListeners {
if err := streamingListener.ListenEndBlock(app.deliverState.ctx, req, res); err != nil {
panic(fmt.Errorf("EndBlock listening hook failed, height: %d, err: %w", req.Height, err))
// call the streaming service hook with the EndBlock messages
for _, abciListener := range app.streamingManager.AbciListeners {
ctx := app.deliverState.ctx
blockHeight := ctx.BlockHeight()
if err := abciListener.ListenEndBlock(ctx, req, res); err != nil {
app.logger.Error("EndBlock listening hook failed", "height", blockHeight, "err", err)
}
}

Expand Down Expand Up @@ -263,14 +269,18 @@ func (app *BaseApp) CheckTx(req abci.RequestCheckTx) abci.ResponseCheckTx {
// Otherwise, the ResponseDeliverTx will contain releveant error information.
// Regardless of tx execution outcome, the ResponseDeliverTx will contain relevant
// gas execution context.
func (app *BaseApp) DeliverTx(req abci.RequestDeliverTx) (res abci.ResponseDeliverTx) {
func (app *BaseApp) DeliverTx(req abci.RequestDeliverTx) abci.ResponseDeliverTx {
gInfo := sdk.GasInfo{}
resultStr := "successful"

var res abci.ResponseDeliverTx
defer func() {
for _, streamingListener := range app.abciListeners {
if err := streamingListener.ListenDeliverTx(app.deliverState.ctx, req, res); err != nil {
panic(fmt.Errorf("DeliverTx listening hook failed: %w", err))
// call the streaming service hook with the EndBlock messages
for _, abciListener := range app.streamingManager.AbciListeners {
ctx := app.deliverState.ctx
blockHeight := ctx.BlockHeight()
if err := abciListener.ListenDeliverTx(ctx, req, res); err != nil {
app.logger.Error("DeliverTx listening hook failed", "height", blockHeight, "err", err)
}
}
}()
Expand All @@ -288,13 +298,15 @@ func (app *BaseApp) DeliverTx(req abci.RequestDeliverTx) (res abci.ResponseDeliv
return sdkerrors.ResponseDeliverTxWithEvents(err, gInfo.GasWanted, gInfo.GasUsed, sdk.MarkEventsToIndex(anteEvents, app.indexEvents), app.trace)
}

return abci.ResponseDeliverTx{
res = abci.ResponseDeliverTx{
GasWanted: int64(gInfo.GasWanted), // TODO: Should type accept unsigned ints?
GasUsed: int64(gInfo.GasUsed), // TODO: Should type accept unsigned ints?
Log: result.Log,
Data: result.Data,
Events: sdk.MarkEventsToIndex(result.Events, app.indexEvents),
}

return res
}

// Commit implements the ABCI interface. It will commit all state that exists in
Expand All @@ -305,6 +317,7 @@ func (app *BaseApp) DeliverTx(req abci.RequestDeliverTx) (res abci.ResponseDeliv
// against that height and gracefully halt if it matches the latest committed
// height.
func (app *BaseApp) Commit() abci.ResponseCommit {

header := app.deliverState.ctx.BlockHeader()
retainHeight := app.GetBlockRetentionHeight(header.Height)

Expand All @@ -319,11 +332,18 @@ func (app *BaseApp) Commit() abci.ResponseCommit {
RetainHeight: retainHeight,
}

// call the hooks with the Commit message
for _, streamingListener := range app.abciListeners {
if err := streamingListener.ListenCommit(app.deliverState.ctx, res); err != nil {
panic(fmt.Errorf("Commit listening hook failed, height: %d, err: %w", header.Height, err))
// call the streaming service hook with the EndBlock messages
abciListeners := app.streamingManager.AbciListeners
if len(abciListeners) > 0 {
ctx := app.deliverState.ctx
blockHeight := ctx.BlockHeight()
changeSet := app.cms.PopStateCache()
for _, abciListener := range abciListeners {
if err := abciListener.ListenCommit(ctx, res, changeSet); err != nil {
app.logger.Error("ListenCommit listening hook failed", "height", blockHeight, "err", err)
}
}
changeSet = nil
}

app.logger.Info("commit synced", "commit", fmt.Sprintf("%X", commitID))
Expand Down Expand Up @@ -376,6 +396,9 @@ func (app *BaseApp) halt() {
}
}

// Stop any running streaming plugins
plugin.CleanupClients()

// Resort to exiting immediately if the process could not be found or killed
// via SIGINT/SIGTERM signals.
app.logger.Info("failed to send SIGINT/SIGTERM; exiting...")
Expand Down
5 changes: 2 additions & 3 deletions baseapp/baseapp.go
Original file line number Diff line number Diff line change
Expand Up @@ -107,9 +107,8 @@ type BaseApp struct { // nolint: maligned
// which informs Tendermint what to index. If empty, all events will be indexed.
indexEvents map[string]struct{}

// abciListeners for hooking into the ABCI message processing of the BaseApp
// and exposing the requests and responses to external consumers
abciListeners []ABCIListener
// streamingManager for managing instances and configuration of ABCIListener services
streamingManager storetypes.StreamingManager

feeHandler sdk.FeeHandler

Expand Down
79 changes: 79 additions & 0 deletions baseapp/baseapp_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2265,3 +2265,82 @@ func TestDefaultStoreLoader(t *testing.T) {
require.Equal(t, storetypes.StoreTypeIAVL, store.GetStoreType())
}
}

var (
distKey1 = storetypes.NewKVStoreKey("distKey1")
)

func TestABCI_MultiListener_StateChanges(t *testing.T) {
// increment the tx counter
anteKey := []byte("ante-key")
anteOpt := func(bapp *BaseApp) { bapp.SetAnteHandler(anteHandlerTxTest(t, capKey1, anteKey)) }

// increment the msg counter
deliverKey := []byte("deliver-key")
deliverKey2 := []byte("deliver-key2")
routerOpt := func(bapp *BaseApp) {
r1 := sdk.NewRoute(routeMsgCounter, handlerMsgCounter(t, capKey1, deliverKey))
r2 := sdk.NewRoute(routeMsgCounter2, handlerMsgCounter(t, capKey1, deliverKey2))
bapp.Router().AddRoute(r1)
bapp.Router().AddRoute(r2)
}

distOpt := func(bapp *BaseApp) { bapp.MountStores(distKey1) }
mockListener1 := NewMockABCIListener("lis_1")
mockListener2 := NewMockABCIListener("lis_2")
streamingManager := storetypes.StreamingManager{AbciListeners: []storetypes.ABCIListener{&mockListener1, &mockListener2}}
streamingManagerOpt := func(bapp *BaseApp) { bapp.SetStreamingManager(streamingManager) }
addListenerOpt := func(bapp *BaseApp) { bapp.CommitMultiStore().AddListeners([]storetypes.StoreKey{distKey1}) }

app := setupBaseApp(t, anteOpt, routerOpt, distOpt, streamingManagerOpt, addListenerOpt)
app.InitChain(abci.RequestInitChain{})

// Create same codec used in txDecoder
codec := codec.NewLegacyAmino()
registerTestCodec(codec)

nBlocks := 3
txPerHeight := 5

for blockN := 0; blockN < nBlocks; blockN++ {
header := tmproto.Header{Height: int64(blockN) + 1}
app.BeginBlock(abci.RequestBeginBlock{Header: header})
var expectedChangeSet []*storetypes.StoreKVPair

for i := 0; i < txPerHeight; i++ {
counter := int64(blockN*txPerHeight + i)
tx := newTxCounter(counter, counter)

txBytes, err := codec.Marshal(tx)
require.NoError(t, err)

sKey := []byte(fmt.Sprintf("distKey%d", i))
sVal := []byte(fmt.Sprintf("distVal%d", i))

ctx := app.getState(runTxModeDeliver).ctx
store := ctx.KVStore(distKey1)
store.Set(sKey, sVal)

expectedChangeSet = append(expectedChangeSet, &storetypes.StoreKVPair{
StoreKey: distKey1.Name(),
Delete: false,
Key: sKey,
Value: sVal,
})

res := app.DeliverTx(abci.RequestDeliverTx{Tx: txBytes})
require.True(t, res.IsOK(), fmt.Sprintf("%v", res))

events := res.GetEvents()
require.Len(t, events, 3, "should contain ante handler, message type and counter events respectively")
require.Equal(t, sdk.MarkEventsToIndex(counterEvent("ante_handler", counter).ToABCIEvents(), map[string]struct{}{})[0], events[0], "ante handler event")
require.Equal(t, sdk.MarkEventsToIndex(counterEvent(sdk.EventTypeMessage, counter).ToABCIEvents(), map[string]struct{}{})[0], events[2], "msg handler update counter event")
}

app.EndBlock(abci.RequestEndBlock{})
app.Commit()

require.Equal(t, expectedChangeSet, mockListener1.ChangeSet, "should contain the same changeSet")
require.Equal(t, expectedChangeSet, mockListener2.ChangeSet, "should contain the same changeSet")
}
}
17 changes: 6 additions & 11 deletions baseapp/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/cosmos/cosmos-sdk/snapshots"
snapshottypes "github.com/cosmos/cosmos-sdk/snapshots/types"
"github.com/cosmos/cosmos-sdk/store"
storetypes "github.com/cosmos/cosmos-sdk/store/types"
sdk "github.com/cosmos/cosmos-sdk/types"
)

Expand Down Expand Up @@ -234,17 +235,6 @@ func (app *BaseApp) SetInterfaceRegistry(registry types.InterfaceRegistry) {
app.msgServiceRouter.SetInterfaceRegistry(registry)
}

// SetStreamingService is used to set a streaming service into the BaseApp hooks and load the listeners into the multistore
func (app *BaseApp) SetStreamingService(s StreamingService) {
// add the listeners for each StoreKey
for key, lis := range s.Listeners() {
app.cms.AddListeners(key, lis)
}
// register the StreamingService within the BaseApp
// BaseApp will pass BeginBlock, DeliverTx, and EndBlock requests and responses to the streaming services to update their ABCI context
app.abciListeners = append(app.abciListeners, s)
}

// SetQueryMultiStore set a alternative MultiStore implementation to support grpc query service.
//
// Ref: https://github.com/cosmos/cosmos-sdk/issues/13317
Expand Down Expand Up @@ -272,3 +262,8 @@ func (app *BaseApp) SetAggregateEventsFunc(aggregateEventsFunc func(resultEvents

app.aggregateEventsFunc = aggregateEventsFunc
}

// SetStreamingManager sets the streaming manager for the BaseApp.test
func (app *BaseApp) SetStreamingManager(manager storetypes.StreamingManager) {
app.streamingManager = manager
}
Loading

0 comments on commit 393db96

Please sign in to comment.