Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Permissionless ICS #2171

Merged
merged 46 commits into from
Sep 6, 2024
Merged

feat!: Permissionless ICS #2171

merged 46 commits into from
Sep 6, 2024

Conversation

mpoke
Copy link
Contributor

@mpoke mpoke commented Aug 26, 2024

Description

Closes: #2108


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • Included the correct type prefix in the PR title
  • Added ! to the type prefix if the change is state-machine breaking
  • Confirmed this PR does not introduce changes requiring state migrations, OR migration code has been added to consumer and/or provider modules
  • Targeted the correct branch (see PR Targeting)
  • Provided a link to the relevant issue or specification
  • Followed the guidelines for building SDK modules
  • Included the necessary unit and integration tests
  • Added a changelog entry to CHANGELOG.md
  • Included comments for documenting Go code
  • Updated the relevant documentation or specification
  • Reviewed "Files changed" and left comments if necessary
  • Confirmed all CI checks have passed
  • If this PR is library API breaking, bump the go.mod version string of the repo, and follow through on a new major release

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! the type prefix if the change is state-machine breaking
  • confirmed this PR does not introduce changes requiring state migrations, OR confirmed migration code has been added to consumer and/or provider modules
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage

testutil/keeper/unit_test_helpers.go Fixed Show fixed Hide fixed
x/ccv/provider/keeper/key_assignment.go Dismissed Show dismissed Hide dismissed
x/ccv/provider/keeper/relay.go Fixed Show fixed Hide fixed
@github-actions github-actions bot added C:Testing Assigned automatically by the PR labeler C:x/provider Assigned automatically by the PR labeler C:x/types Assigned automatically by the PR labeler C:Build Assigned automatically by the PR labeler C:Docs Assigned automatically by the PR labeler C:ADR Assigned automatically by the PR labeler labels Aug 26, 2024
insumity and others added 8 commits August 28, 2024 14:58
* (partially) renamed chain ids to consumer ids

* renamed proposal messages

* removed global slash entry

* fixed unit tests

* added new messages

* introduced new state

* added functionality for the register and initialize messages

* renamed (partially) chainIds to consumerIds

* set consumerId to chainId association during registration

* added extra check in the initialization so unknokwn, launched, or stopped chains cannot re-initialize

* added initial work on traversing initialized chains that are to-be-launched

* fixed rebase issues after bringing the VSCMaturedPackets work in

* made it so we traverse initialization records instead of addition proposals (+ additional changes so the unit tests pass)

* renamed more chainIDs to consumerIds

* removed ClientIdToChainId state because chainId already resides on the registration record

* nit fixes in go docs

* removed MsgConsumerAddition

* added CLI commands for new messages

* removed consumer modification proposal

* removed (partially) consumer removal proposal

* rebased to pick up the inactive-validators work (PR #2079)

* introduced consumerId in the equivocation messages (and a useful query for Hermes to get the consumerId)

* added safeguard so that a validator cannot opt-in to two different chains with the same chain id

* renamed some chainIDs to consumerIds

* updated based on comments

Co-authored-by: bernd-m <[email protected]>

* fixed integration tests

* rebased to pick up the removal of legacy proposals (#2130) and re-introduced old messages so that existing proposals can deserialize

* changes messages to only have MsgCreateConsumer and MsgUpdateConsumer and modified protos so that we are backward-compatible

* cleaned up slightly a few things (mostly committing & pushing) so people can pick up the latest changes

* fixed the CreateConsumer and UpdateConsumer logic and made most of the fields optional

* fixed hooks and the code around proposalId to consumerId

* feat: extend consumer validator query to return commission rate (backport #2162) (#2165)

* adapt #2162 changes for permissionless ICS

* nits

---------

Co-authored-by: kirdatatjana <[email protected]>

* renamed some chainIds to consumerIds

* took into account comments and also added safeguard to reject new proposals that still use deprecated messages (e.g., MsgConsumerAddition, etc.)

* Update x/ccv/provider/types/msg.go

Co-authored-by: bernd-m <[email protected]>

* removed double-gas charge on MsgCreateConsumer and imroved the logic of MsgUpdateConsumer

* added PopulateMinimumPowerInTopN tested

* took into account comments (using protos for marshalling string slice, fixed issues in the UpdateConsumer logic, added extra check to abort spurious proposals)

* feat: add fields to consumer validators query (#2167)

* extend consumer validators query

* nit

* nits

* fix msg order

* deprecate power for consumer_power

* modified the way we verify the new owner address, as well as nit refactoring on the ConsumerIds

* fixed some rebase issues and changed a proto to be backward-compatible

---------

Co-authored-by: bernd-m <[email protected]>
Co-authored-by: Simon Noetzlin <[email protected]>
Co-authored-by: kirdatatjana <[email protected]>
* add UT

* nits

* address comments

* Update x/ccv/provider/keeper/partial_set_security.go

Co-authored-by: insumity <[email protected]>

* fix tests

---------

Co-authored-by: insumity <[email protected]>
…2176)

ConsumerModificationProposal is needed in Gaia upgrade handler
mpoke and others added 5 commits August 28, 2024 15:54
…fore launch (#2164)

* (partially) renamed chain ids to consumer ids

* renamed proposal messages

* removed global slash entry

* fixed unit tests

* added new messages

* introduced new state

* added functionality for the register and initialize messages

* renamed (partially) chainIds to consumerIds

* set consumerId to chainId association during registration

* added extra check in the initialization so unknokwn, launched, or stopped chains cannot re-initialize

* added initial work on traversing initialized chains that are to-be-launched

* fixed rebase issues after bringing the VSCMaturedPackets work in

* made it so we traverse initialization records instead of addition proposals (+ additional changes so the unit tests pass)

* renamed more chainIDs to consumerIds

* removed ClientIdToChainId state because chainId already resides on the registration record

* nit fixes in go docs

* removed MsgConsumerAddition

* added CLI commands for new messages

* removed consumer modification proposal

* removed (partially) consumer removal proposal

* rebased to pick up the inactive-validators work (PR #2079)

* introduced consumerId in the equivocation messages (and a useful query for Hermes to get the consumerId)

* added safeguard so that a validator cannot opt-in to two different chains with the same chain id

* renamed some chainIDs to consumerIds

* updated based on comments

Co-authored-by: bernd-m <[email protected]>

* fixed integration tests

* rebased to pick up the removal of legacy proposals (#2130) and re-introduced old messages so that existing proposals can deserialize

* changes messages to only have MsgCreateConsumer and MsgUpdateConsumer and modified protos so that we are backward-compatible

* cleaned up slightly a few things (mostly committing & pushing) so people can pick up the latest changes

* fixed the CreateConsumer and UpdateConsumer logic and made most of the fields optional

* fixed hooks and the code around proposalId to consumerId

* pre-spawn query

* rebase

* nits

* feat: extend consumer validator query to return commission rate (backport #2162) (#2165)

* adapt #2162 changes for permissionless ICS

* nits

---------

Co-authored-by: kirdatatjana <[email protected]>

* remove panics

* renamed some chainIds to consumerIds

* took into account comments and also added safeguard to reject new proposals that still use deprecated messages (e.g., MsgConsumerAddition, etc.)

* Update x/ccv/provider/types/msg.go

Co-authored-by: bernd-m <[email protected]>

* removed double-gas charge on MsgCreateConsumer and imroved the logic of MsgUpdateConsumer

* added PopulateMinimumPowerInTopN tested

* took into account comments (using protos for marshalling string slice, fixed issues in the UpdateConsumer logic, added extra check to abort spurious proposals)

* update logic

* took into account comments (using protos for marshalling string slice, fixed issues in the UpdateConsumer logic, added extra check to abort spurious proposals)

* feat: add fields to consumer validators query (#2167)

* extend consumer validators query

* nit

* nits

* fix msg order

* deprecate power for consumer_power

* nits

* feat: first iteration on Permissionless ICS (#2117)

* (partially) renamed chain ids to consumer ids

* renamed proposal messages

* removed global slash entry

* fixed unit tests

* added new messages

* introduced new state

* added functionality for the register and initialize messages

* renamed (partially) chainIds to consumerIds

* set consumerId to chainId association during registration

* added extra check in the initialization so unknokwn, launched, or stopped chains cannot re-initialize

* added initial work on traversing initialized chains that are to-be-launched

* fixed rebase issues after bringing the VSCMaturedPackets work in

* made it so we traverse initialization records instead of addition proposals (+ additional changes so the unit tests pass)

* renamed more chainIDs to consumerIds

* removed ClientIdToChainId state because chainId already resides on the registration record

* nit fixes in go docs

* removed MsgConsumerAddition

* added CLI commands for new messages

* removed consumer modification proposal

* removed (partially) consumer removal proposal

* rebased to pick up the inactive-validators work (PR #2079)

* introduced consumerId in the equivocation messages (and a useful query for Hermes to get the consumerId)

* added safeguard so that a validator cannot opt-in to two different chains with the same chain id

* renamed some chainIDs to consumerIds

* updated based on comments

Co-authored-by: bernd-m <[email protected]>

* fixed integration tests

* rebased to pick up the removal of legacy proposals (#2130) and re-introduced old messages so that existing proposals can deserialize

* changes messages to only have MsgCreateConsumer and MsgUpdateConsumer and modified protos so that we are backward-compatible

* cleaned up slightly a few things (mostly committing & pushing) so people can pick up the latest changes

* fixed the CreateConsumer and UpdateConsumer logic and made most of the fields optional

* fixed hooks and the code around proposalId to consumerId

* feat: extend consumer validator query to return commission rate (backport #2162) (#2165)

* adapt #2162 changes for permissionless ICS

* nits

---------

Co-authored-by: kirdatatjana <[email protected]>

* renamed some chainIds to consumerIds

* took into account comments and also added safeguard to reject new proposals that still use deprecated messages (e.g., MsgConsumerAddition, etc.)

* Update x/ccv/provider/types/msg.go

Co-authored-by: bernd-m <[email protected]>

* removed double-gas charge on MsgCreateConsumer and imroved the logic of MsgUpdateConsumer

* added PopulateMinimumPowerInTopN tested

* took into account comments (using protos for marshalling string slice, fixed issues in the UpdateConsumer logic, added extra check to abort spurious proposals)

* feat: add fields to consumer validators query (#2167)

* extend consumer validators query

* nit

* nits

* fix msg order

* deprecate power for consumer_power

* modified the way we verify the new owner address, as well as nit refactoring on the ConsumerIds

* fixed some rebase issues and changed a proto to be backward-compatible

---------

Co-authored-by: bernd-m <[email protected]>
Co-authored-by: Simon Noetzlin <[email protected]>
Co-authored-by: kirdatatjana <[email protected]>

* cover stopped phase case

* fixed bug on removing previous spawn time & added tests

* added some additional tests

* added tests on the hooks

* removed check that spawn time is in the future

* feat: refactor consumer validator set computation (#2175)

* add UT

* nits

* address comments

* Update x/ccv/provider/keeper/partial_set_security.go

Co-authored-by: insumity <[email protected]>

* fix tests

---------

Co-authored-by: insumity <[email protected]>

* nit

* nits

* nit

---------

Co-authored-by: insumity <[email protected]>
Co-authored-by: bernd-m <[email protected]>
Co-authored-by: kirdatatjana <[email protected]>
* fix integration test setup

* fix integration tests
testutil/keeper/unit_test_helpers.go Dismissed Show dismissed Hide dismissed
sainoe and others added 5 commits August 29, 2024 09:42
* feat: first iteration on Permissionless ICS (#2117)

* (partially) renamed chain ids to consumer ids

* renamed proposal messages

* removed global slash entry

* fixed unit tests

* added new messages

* introduced new state

* added functionality for the register and initialize messages

* renamed (partially) chainIds to consumerIds

* set consumerId to chainId association during registration

* added extra check in the initialization so unknokwn, launched, or stopped chains cannot re-initialize

* added initial work on traversing initialized chains that are to-be-launched

* fixed rebase issues after bringing the VSCMaturedPackets work in

* made it so we traverse initialization records instead of addition proposals (+ additional changes so the unit tests pass)

* renamed more chainIDs to consumerIds

* removed ClientIdToChainId state because chainId already resides on the registration record

* nit fixes in go docs

* removed MsgConsumerAddition

* added CLI commands for new messages

* removed consumer modification proposal

* removed (partially) consumer removal proposal

* rebased to pick up the inactive-validators work (PR #2079)

* introduced consumerId in the equivocation messages (and a useful query for Hermes to get the consumerId)

* added safeguard so that a validator cannot opt-in to two different chains with the same chain id

* renamed some chainIDs to consumerIds

* updated based on comments

Co-authored-by: bernd-m <[email protected]>

* fixed integration tests

* rebased to pick up the removal of legacy proposals (#2130) and re-introduced old messages so that existing proposals can deserialize

* changes messages to only have MsgCreateConsumer and MsgUpdateConsumer and modified protos so that we are backward-compatible

* cleaned up slightly a few things (mostly committing & pushing) so people can pick up the latest changes

* fixed the CreateConsumer and UpdateConsumer logic and made most of the fields optional

* fixed hooks and the code around proposalId to consumerId

* feat: extend consumer validator query to return commission rate (backport #2162) (#2165)

* adapt #2162 changes for permissionless ICS

* nits

---------

Co-authored-by: kirdatatjana <[email protected]>

* renamed some chainIds to consumerIds

* took into account comments and also added safeguard to reject new proposals that still use deprecated messages (e.g., MsgConsumerAddition, etc.)

* Update x/ccv/provider/types/msg.go

Co-authored-by: bernd-m <[email protected]>

* removed double-gas charge on MsgCreateConsumer and imroved the logic of MsgUpdateConsumer

* added PopulateMinimumPowerInTopN tested

* took into account comments (using protos for marshalling string slice, fixed issues in the UpdateConsumer logic, added extra check to abort spurious proposals)

* feat: add fields to consumer validators query (#2167)

* extend consumer validators query

* nit

* nits

* fix msg order

* deprecate power for consumer_power

* modified the way we verify the new owner address, as well as nit refactoring on the ConsumerIds

* fixed some rebase issues and changed a proto to be backward-compatible

---------

Co-authored-by: bernd-m <[email protected]>
Co-authored-by: Simon Noetzlin <[email protected]>
Co-authored-by: kirdatatjana <[email protected]>

* fixed bug on removing previous spawn time & added tests

* added some additional tests

* added tests on the hooks

* removed check that spawn time is in the future

* feat: refactor consumer validator set computation (#2175)

* add UT

* nits

* address comments

* Update x/ccv/provider/keeper/partial_set_security.go

Co-authored-by: insumity <[email protected]>

* fix tests

---------

Co-authored-by: insumity <[email protected]>

* add UT

* nits

* nits

* revert legacy prop funcs

* fix UT

---------

Co-authored-by: insumity <[email protected]>
Co-authored-by: bernd-m <[email protected]>
Co-authored-by: kirdatatjana <[email protected]>
* feat: first iteration on Permissionless ICS (#2117)

* (partially) renamed chain ids to consumer ids

* renamed proposal messages

* removed global slash entry

* fixed unit tests

* added new messages

* introduced new state

* added functionality for the register and initialize messages

* renamed (partially) chainIds to consumerIds

* set consumerId to chainId association during registration

* added extra check in the initialization so unknokwn, launched, or stopped chains cannot re-initialize

* added initial work on traversing initialized chains that are to-be-launched

* fixed rebase issues after bringing the VSCMaturedPackets work in

* made it so we traverse initialization records instead of addition proposals (+ additional changes so the unit tests pass)

* renamed more chainIDs to consumerIds

* removed ClientIdToChainId state because chainId already resides on the registration record

* nit fixes in go docs

* removed MsgConsumerAddition

* added CLI commands for new messages

* removed consumer modification proposal

* removed (partially) consumer removal proposal

* rebased to pick up the inactive-validators work (PR #2079)

* introduced consumerId in the equivocation messages (and a useful query for Hermes to get the consumerId)

* added safeguard so that a validator cannot opt-in to two different chains with the same chain id

* renamed some chainIDs to consumerIds

* updated based on comments

Co-authored-by: bernd-m <[email protected]>

* fixed integration tests

* rebased to pick up the removal of legacy proposals (#2130) and re-introduced old messages so that existing proposals can deserialize

* changes messages to only have MsgCreateConsumer and MsgUpdateConsumer and modified protos so that we are backward-compatible

* cleaned up slightly a few things (mostly committing & pushing) so people can pick up the latest changes

* fixed the CreateConsumer and UpdateConsumer logic and made most of the fields optional

* fixed hooks and the code around proposalId to consumerId

* feat: extend consumer validator query to return commission rate (backport #2162) (#2165)

* adapt #2162 changes for permissionless ICS

* nits

---------

Co-authored-by: kirdatatjana <[email protected]>

* renamed some chainIds to consumerIds

* took into account comments and also added safeguard to reject new proposals that still use deprecated messages (e.g., MsgConsumerAddition, etc.)

* Update x/ccv/provider/types/msg.go

Co-authored-by: bernd-m <[email protected]>

* removed double-gas charge on MsgCreateConsumer and imroved the logic of MsgUpdateConsumer

* added PopulateMinimumPowerInTopN tested

* took into account comments (using protos for marshalling string slice, fixed issues in the UpdateConsumer logic, added extra check to abort spurious proposals)

* feat: add fields to consumer validators query (#2167)

* extend consumer validators query

* nit

* nits

* fix msg order

* deprecate power for consumer_power

* modified the way we verify the new owner address, as well as nit refactoring on the ConsumerIds

* fixed some rebase issues and changed a proto to be backward-compatible

---------

Co-authored-by: bernd-m <[email protected]>
Co-authored-by: Simon Noetzlin <[email protected]>
Co-authored-by: kirdatatjana <[email protected]>

* add phase + metadata

* first logic draft

* add filter

* reformat test

* nits

* nit

* address comments

* update tests

* nits

* update logic

* update CLI

* revert unwanted changes

* remove filter field

* nit

* fix bad int conversion

* update cli

---------

Co-authored-by: insumity <[email protected]>
Co-authored-by: bernd-m <[email protected]>
Co-authored-by: kirdatatjana <[email protected]>
* migrations for permissionless -- wip

* feat!: migrations for permissionless (#2177)

* initial commit

* took into account comments

* removed deprecated queries

* fix error due to rebase

* remove fields from provider genesis

* remove commented code

---------

Co-authored-by: insumity <[email protected]>
update consumer chains REST endpoint
* add consumer registration event

* fix some queries, CLI and permissionless logic

* cli changes

* addressed review comments
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bermuell This entire file needs to be removed. This will affect the e2e tests though.

@mpoke mpoke enabled auto-merge September 5, 2024 15:19
sainoe and others added 3 commits September 5, 2024 19:24
* endpoint fixes

* Update proto/interchain_security/ccv/provider/v1/query.proto

Co-authored-by: MSalopek <[email protected]>

* Update proto/interchain_security/ccv/provider/v1/query.proto

Co-authored-by: MSalopek <[email protected]>

* generate proto

---------

Co-authored-by: MSalopek <[email protected]>
replace CanLaunch with InitializeConsumer
Copy link
Contributor

@insumity insumity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @mpoke. I did not review but accepting (✅) to unblock cutting. I'll review at a later stage.

Copy link
Contributor

@MSalopek MSalopek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Admin approval.
Review will follow.

@mpoke mpoke added this pull request to the merge queue Sep 6, 2024
Merged via the queue into main with commit 7a22a17 Sep 6, 2024
22 of 26 checks passed
x/ccv/provider/handler.go Show resolved Hide resolved
x/ccv/provider/handler_test.go Show resolved Hide resolved
x/ccv/provider/proposal_handler.go Show resolved Hide resolved
x/ccv/provider/types/throttle.go Show resolved Hide resolved
string name = 1;
// the description of the chain
string description = 2;
// the metadata (e.g., GitHub repository URL) of the chain
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make a comment that this is just a string. It can be a JSON string etc. The chain shouldnot care if it's a URL or not.

@p-offtermatt
Copy link
Contributor

Also outdated here:

// GetTemplateClient returns the template client for provider proposals
func (k Keeper) GetTemplateClient(ctx sdk.Context) *ibctmtypes.ClientState {
params := k.GetParams(ctx)
return params.TemplateClient
}

@mpoke
Copy link
Contributor Author

mpoke commented Sep 9, 2024

@p-offtermatt I addressed comments until here in #2237

@p-offtermatt
Copy link
Contributor

p-offtermatt commented Sep 9, 2024

Note: Already discussed with @insumity about this one

There is a bug around the spawntime in UpdateConsumer; essentially it's possible fo a user to set a consumer chain that will attempt to launch in every block.

Assume we have a Registered consumer chain with a SpawnTime.
Now, a user sends a MsgUpdateConsumer with InitializationParameters where SpawnTime==0.
(This is not validated against in ValidateBasic

func ValidateInitializationParameters(initializationParameters ConsumerInitializationParameters) error {
)

The new InitializationParameters get set here:

if err = k.Keeper.SetConsumerInitializationParameters(ctx, msg.ConsumerId, *msg.InitializationParameters); err != nil {

However, the InitializeConsumer call here

if spawnTime, initialized := k.Keeper.InitializeConsumer(ctx, consumerId); initialized {
will fail because the SpawnTime is zero, see this check
if initializationParameters.SpawnTime.IsZero() {

Thus, the PrepareConsumerForLaunch call here

if err := k.Keeper.PrepareConsumerForLaunch(ctx, consumerId, previousSpawnTime, spawnTime); err != nil {
does not update the SpawnTime in the ConsumerToBeLaunched store.

So the InitializationParameters will say SpawnTime==0, but the store will have SpawnTime=oldSpawnTime.

During BeginBlock, when the consumers are launched, this RemoveConsumerToBeLaunched call here will thus not remove the consumer

err = k.RemoveConsumerToBeLaunched(ctx, consumerId, initializationParameters.SpawnTime)
and error, which causes the loop to continue.

In the extreme, someone can create 200 such consumers, and those will be re-tried to be launched in every block and no other consumers will ever launch

@p-offtermatt
Copy link
Contributor

p-offtermatt commented Sep 9, 2024

Another find:
The owner can update launched consumer chains; in particular the owner can arbitrarily modify the InitializationParameters of the already launched chain. afaict this doesn't have any further impact, but it means the InitializationParameters returned by QueryConsumerChain can be arbitrary, and it would look like the chain launched with different parameters than it actually launched with (i.e. could make it look like the chain had a very large ConsumerRedistributionFraction)

I think a fix would be to just not let the InitializationParameters of launched consumer chains be modified

@mpoke
Copy link
Contributor Author

mpoke commented Sep 9, 2024

So the InitializationParameters will say SpawnTime==0, but the store will have SpawnTime=oldSpawnTime.

During BeginBlock, when the consumers are launched, this RemoveConsumerToBeLaunched call here will thus not remove the consumer

err = k.RemoveConsumerToBeLaunched(ctx, consumerId, initializationParameters.SpawnTime)

and error, which causes the loop to continue.
In the extreme, someone can create 200 such consumers, and those will be re-tried to be launched in every block and no other consumers will ever launch

Nice catch @p-offtermatt. I do think that #2236 somehow fixed this, but we should probably not have inconsistencies between the initialization params and the time queues used to store the chains to be launched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C:ADR Assigned automatically by the PR labeler C:Build Assigned automatically by the PR labeler C:Docs Assigned automatically by the PR labeler C:Testing Assigned automatically by the PR labeler C:x/provider Assigned automatically by the PR labeler C:x/types Assigned automatically by the PR labeler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EPIC: Permissionless ICS
6 participants