feat: add runtime Env to icacontroller and fix v0.52 e2e tests #7587

damiannolan · 2024-11-21T17:13:30Z

Description

Initially adding runtime Environment to ica to address events propagation issue with unwrapping context.
All tests seem to be now passing on this branch, ~~except for some build err in e2e unit tests which we will address separately~~.

Update: E2E and unit tests passing on this branch. Issues created for follow up items.

ref: #7223

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

damiannolan · 2024-11-25T11:34:08Z

Maybe we need to update linter versions or something in CI. Not sure what the arm64 build failure is too. Probably need to look into it

…ypes

damiannolan · 2024-12-02T10:24:41Z

Module safe query e2e test via interchain accounts is failing with:

 key: ibccallbackerror-error
 value: 'failed to execute interchain account transaction: rpc error: code = InvalidArgument
              desc = invalid address: decoding bech32 failed: string not all lowercase or
              all uppercase'

This looks like something is malformed when executing the interchain accounts tx via MsgRecvPacket

edit: looks like its coming from here -> https://github.com/cosmos/cosmos-sdk/blob/main/x/bank/keeper/grpc_query.go#L40-L43

Fixed in f9b7314. Looks like some address change crept in here. There was a TODO inline which needed to be fixed. The address must be bech32.

damiannolan · 2024-12-02T11:30:59Z

Remaining test failures:

IBCGenesisTest (for export state) is failing on CI but this test passes locally. Its failing with some docker connection refused errors when attempting to restart all nodes. Not sure on best path for fixing this atm.

Solved: InitChain was missing a sorting of validators looked up from the statemachine. cosmos/cosmos-sdk#22757 ✅

TestClientRecovery is failing reporting that the subject client is still active. I have run this test locally and exec'd into the chainA container and run cli queries for client status where the client (07-tendermint-1) is reported correctly as expired. Very strange behaviour...

Solved: Query context was missing time via header info. cosmos/cosmos-sdk#22732 ✅

TestScheduleIBCUpgrade is failing due to some protobuf Any encoding error. I have not yet debugged this in detail but also seems to be quite odd. The test fails x/gov proposal with the following:

cannot unpack client state: cannot unpack Any into ClientState *types.Any: failed unpacking protobuf message from Any: invalid client type: failed waiting for condition after 30.000000 seconds

Other than this its just linters that need to be fixed.. which looks like another can of worms

…rshaler for Codec iface

damiannolan · 2024-12-04T10:11:14Z

Update on 1. above. The TestIBCGenesis failure is reproducible locally by using multiple validator and full nodes. The error occurs when restarting all nodes.

ERR failure when running app err="error during handshake: error on replay: genesisValidators[0] != req.Validators[0] "
    setup.go:124:

I think the failure is possibly related to export.go in simapp. Particularly the ExportAppStateAndValidators func. We can look into it there

Solved: InitChain was missing a sorting of validators looked up from the statemachine. cosmos/cosmos-sdk#22757 ✅

… main

…de, update code to expect tendermint client temporarily

modules/core/keeper/msg_server.go

damiannolan · 2024-12-05T16:50:02Z

modules/core/02-client/types/codec.go

+		// TODO(damian): revert this extra err info
+		return nil, errorsmod.Wrapf(ibcerrors.ErrUnpackAny, "cannot unpack Any into ClientState %T, typeURL: %s, cachedValue: %v", protoAny, protoAny.TypeUrl, protoAny.GetCachedValue())


Revert this

Does it need an issue to make sure we don't forget, or will you remember? :)

damiannolan · 2024-12-05T16:50:47Z

modules/core/02-client/keeper/keeper_test.go

@@ -642,3 +643,59 @@ func (suite *KeeperTestSuite) TestIBCSoftwareUpgrade() {
 		})
 	}
 }
+
+func (suite *KeeperTestSuite) TestIBCScheduledUpgradeProposal() {


This can also be deleted I think once cosmos/cosmos-sdk#22779 is solved, but its also no harm keeping around as it routes a msg for scheduleIBCUpgrade through gov

Maybe a comment with a link to that for context?

damiannolan · 2024-12-05T16:53:08Z

modules/apps/27-interchain-accounts/controller/keeper/events.go

+	if err := k.EventService.EventManager(ctx).EmitKV(icatypes.EventTypePacket, attributes...); err != nil {
+		panic(err)
+	}


This error should be propagated and returned from this func.

The likelihood of these errors surfacing from EventManager are pretty much zero, but at some point in a future far away it may be possible for it to error - given a different impl of event service.

None the less, its good practise to actually return the errors and not panic and rely on grpc recovery interceptor

Should it have a todo comment for that, then?

gjermundgaraba

LGTM! Left a couple of questions and minor comments, but looks great to me otherwise 🔥

.github/workflows/callbacks.yml

gjermundgaraba · 2024-12-05T17:14:21Z

e2e/tests/upgrades/upgrade_test.go

@@ -74,22 +74,13 @@ func (s *UpgradeTestSuite) UpgradeChain(ctx context.Context, chain *cosmos.Cosmo
 		Info:   fmt.Sprintf("upgrade version test from %s to %s", currentVersion, upgradeVersion),
 	}

-	if testvalues.GovV1MessagesFeatureReleases.IsSupported(chain.Config().Images[0].Version) {


Is this just not needed anymore, or will this break compatibility tests in any way we need to deal with later?

Pretty sure it should be fine. As 0.47 (ibc-go/v7) has govv1 msg execution.

gjermundgaraba · 2024-12-05T17:15:42Z

e2e/testsuite/testsuite.go

@@ -558,6 +558,18 @@ func (s *E2ETestSuite) createWalletOnChainIndex(ctx context.Context, amount, cha
 	wallet := interchaintest.GetAndFundTestUsers(s.T(), ctx, strings.ReplaceAll(s.T().Name(), " ", "-"), sdkmath.NewInt(amount), chain)[0]
 	// note the GetAndFundTestUsers requires the caller to wait for some blocks before the funds are accessible.
 	s.Require().NoError(test.WaitForBlocks(ctx, 2, chain))
+


💀 ooof

Is this the only way to do this now?

SendCoins to an account no longer creates the account in x/auth. It just stores the coins to the address space in x/bank.

The account only gets created when there has been an action/tx made with the key.
You could probably exec any kind of tx here but this is what @chatton done and it works.

Instances where this came up in unit tests (where SendCoins to account was being used to create an account) I just swapped to directly using the x/auth keeper but that's only possible in unit testing where you have that kind of control

gjermundgaraba · 2024-12-05T17:17:22Z

modules/apps/27-interchain-accounts/controller/keeper/events.go

+	if err := k.EventService.EventManager(ctx).EmitKV(icatypes.EventTypePacket, attributes...); err != nil {
+		panic(err)
+	}


Should it have a todo comment for that, then?

gjermundgaraba · 2024-12-05T17:18:41Z

modules/apps/27-interchain-accounts/controller/keeper/migrations_test.go

@@ -29,11 +31,10 @@ func (suite *KeeperTestSuite) TestMigratorMigrateParams() {
 			func() {
 				suite.chainA.GetSimApp().ICAControllerKeeper = icacontrollerkeeper.NewKeeper(
 					suite.chainA.Codec,
-					runtime.NewKVStoreService(suite.chainA.GetSimApp().GetKey(icacontrollertypes.StoreKey)),
+					runtime.NewEnvironment(runtime.NewKVStoreService(suite.chainA.GetSimApp().GetKey(icacontrollertypes.StoreKey)), log.NewNopLogger()),


I guess we'll have to do some upgrade doc changes for this. Are we tracking that anywhere?

Typically in an epic we'd have for an sdk upgrade we'd have a bullet point for keeping track of this.

gjermundgaraba · 2024-12-05T17:19:16Z

modules/core/02-client/keeper/keeper_test.go

@@ -642,3 +643,59 @@ func (suite *KeeperTestSuite) TestIBCSoftwareUpgrade() {
 		})
 	}
 }
+
+func (suite *KeeperTestSuite) TestIBCScheduledUpgradeProposal() {


Maybe a comment with a link to that for context?

modules/core/keeper/msg_server.go

DimitrisJim

Quickly skimmed, looking good. Trusting the tests. Thanks for shifting through the upgrade swamp 🙏

sonarqubecloud · 2024-12-06T08:56:27Z

Quality Gate passed for 'ibc-go'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

* go mod changes * remove cometabci direct dep * fix all errors * appmodule & simapp cleanup * 08wasm fixes * get 08-wasm compiling * another simapp * another simapp * pass in environment * fix compilation errors * some fixes * build * fix other simapps * linting * remove unsed code * go mod changes * add pool module * anothersimapp * linting fixes * protocol pool module accounts * callback fixes * simapp fixes * register interfaces * fix feeibc * changes++ * fix missed conflict * fix wasm client * linting fixes * fix tests * linting * fix simapp tests * fix some tests * fix e2e go mod * go mod changes * remove grandpa * Gomod changes and minor updates (#7549) * Remove module.HasProposalMsgs * Update ante handler * Bump versions * x * Cleanup * chore: use same tags for all go mods * chore: update go mod files, revert callbacks simapp changes * chore: fix tendermint client test suite * chore: fix tests in TestAnteDecoratorCheckTx * chore: fix event tests * chore: fix compiler error in simapp * deps: update to latest of cosmossdk.io/store main to fix proofs test * chore: tidy all * fix: remove ibcmock module acc from blocked addrs * chore: fix proto failures and Gov errors * chore: fix ica test * chore: fix a few compiler errs in 08-wasm * fix: update ica account already exists test to front run account creation correctly * chore: fix ica test for address gen * chore: re-add log to returned error * chore: update go mods * chore: fixed build errors in wasm app.go * chore: cleanup callbacks simapp * chore: fixing simulation functions * chore: reconfigure simd cmd and fix TestInitCmd * feat: add runtime Env to icacontroller and fix v0.52 e2e tests (#7587) * feat: adding runtime Environment * chore: run make lint-fix * chore: remove duplicate import * chore: rm duplicate import * chore: migrate to env KVStoreService in icacontroller * chore: fix duplicate import * chore: rm kv store service * chore: refactor events to use environment * chore: make lint-fix * chore: make tidy-all * chore: rm msg service router from icacontroller * fix: remove 32 bit arm arch from callbacks ci builds * test: try cache false for callbacks linter * chore: pin to updated interchaintest fork * chore: revert relayer count to 10 * chore: continued work on upgrading e2e tests to 0.52 * chore: transfer e2e passing * chore: fix upgrade build errors * chore: bump interchaintest * chore: update sample config * chore: reconfigure protoany to use gogoproto any in favour of codec types * chore: tidy imports in e2e * fix: address failure in module safe query ica e2e test * bump golangci version to 1.60 * lint: remove duplicate import * chore: use gogoprotoany in favour of codec types any. rm ProtoCodecMarshaler for Codec iface * chore: bump go version in Dockerfile * chore: add todo * lint: fix unnecessary cast * deps: upgrade sdk to head of release/v0.52.x, upgrade store to latest main * deps: bump to head of release/v0.52 * fix: add unit test to reproduce e2e test failure for ScheduleIBCUpgrade, update code to expect tendermint client temporarily * fix: error in tests * chore: address comments from pr --------- Co-authored-by: chatton <[email protected]> Co-authored-by: Nikolas De Giorgis <[email protected]> * chore: linter fixes * lint: ignore redunant import alias * chore: rm redudant semi-colon in tendermint.proto * chore: rm redundant doc string regarding e2e-compatibility tests - tests now use annoations for compatibility checks * chore: update dead links * fix: apply fix for x/tx cosmos-sdk #22574 * chore: make lint-fix * fix: use hermes v1.10.4 for ci configs * fix: apply fix for cosmos-sdk #22779 and update dependencies * fix: unpin local sdk to HEAD of release/v0.52 in simapp go.mod * chore: revert docs/versioned_docs to main * chore: ammendments to linter configs * chore: update var name in expected keeper * chore: revert diffs in clienttypes/height_test * chore: propagate error from event emission in icacontroller * test: revert unnecessary diffs in test fn * chore: update comment in test chain * Update e2e/tests/interchain_accounts/base_test.go Co-authored-by: DimitrisJim <[email protected]> * chore: nit on import aliasing * refactor: rename instances of expected keepers from accountKeeper -> authKeeper where apt * chore: readd TestABCICodeDeterminism to 04-channel types * refactor: remove redundant AppModuleBasics * chore: rm GetStakingKeeper api from testing app * chore: update version in wasm dockerfile * chore: update the go work example file version * chore: rm UnwrapSDKContext in ica where possible * fix: import path in e2e from suggestion --------- Co-authored-by: Alexander Peters <[email protected]> Co-authored-by: chatton <[email protected]> Co-authored-by: Damian Nolan <[email protected]> Co-authored-by: Nikolas De Giorgis <[email protected]> Co-authored-by: DimitrisJim <[email protected]>

damiannolan added 11 commits November 21, 2024 18:13

feat: adding runtime Environment

346d83d

chore: run make lint-fix

29ce5f2

chore: remove duplicate import

7e7d17d

chore: rm duplicate import

7e2f699

chore: migrate to env KVStoreService in icacontroller

7cb2468

chore: fix duplicate import

cb93c24

chore: rm kv store service

f828f09

chore: refactor events to use environment

d488054

chore: make lint-fix

92ecec1

chore: make tidy-all

4660322

chore: rm msg service router from icacontroller

e267e42

damiannolan added 2 commits November 25, 2024 16:19

fix: remove 32 bit arm arch from callbacks ci builds

ab1f3f3

test: try cache false for callbacks linter

8b67d8d

damiannolan changed the title ~~feat: adding runtime Environment~~ feat: adding runtime Environment to icacontroller Nov 25, 2024

chatton added 5 commits November 26, 2024 11:20

chore: pin to updated interchaintest fork

e27966b

chore: revert relayer count to 10

b6fba89

chore: continued work on upgrading e2e tests to 0.52

c23234d

chore: transfer e2e passing

d83116d

chore: fix upgrade build errors

82c5a66

chatton marked this pull request as ready for review November 27, 2024 12:02

chatton requested review from AdityaSripal, srdtrk, chatton, DimitrisJim, gjermundgaraba and bznein as code owners November 27, 2024 12:02

chatton changed the title ~~feat: adding runtime Environment to icacontroller~~ [WIP] feat: adding runtime Environment to icacontroller Nov 27, 2024

chatton added 2 commits November 27, 2024 13:59

chore: bump interchaintest

06d40e4

chore: update sample config

a128c03

damiannolan added 2 commits November 27, 2024 16:41

chore: reconfigure protoany to use gogoproto any in favour of codec t…

55b4fbe

…ypes

chore: tidy imports in e2e

b92bc67

fix: address failure in module safe query ica e2e test

f9b7314

bznein and others added 4 commits December 2, 2024 15:45

bump golangci version to 1.60

94a4685

lint: remove duplicate import

4f37cf6

chore: use gogoprotoany in favour of codec types any. rm ProtoCodecMa…

e24c60f

…rshaler for Codec iface

chore: bump go version in Dockerfile

d70f0c4

damiannolan added 6 commits December 4, 2024 15:10

chore: add todo

5b89030

lint: fix unnecessary cast

5ea5802

deps: upgrade sdk to head of release/v0.52.x, upgrade store to latest…

0e64412

… main

deps: bump to head of release/v0.52

a0f28fc

fix: add unit test to reproduce e2e test failure for ScheduleIBCUpgra…

785dc47

…de, update code to expect tendermint client temporarily

fix: error in tests

9057fcb

damiannolan changed the title ~~[WIP] feat: adding runtime Environment to icacontroller~~ feat: add runtime Env to icacontroller and fix v0.52 e2e tests Dec 5, 2024

damiannolan commented Dec 5, 2024

View reviewed changes

gjermundgaraba approved these changes Dec 5, 2024

View reviewed changes

DimitrisJim approved these changes Dec 6, 2024

View reviewed changes

This was referenced Dec 6, 2024

Propagate events errors in v0.52 icacontroller #7624

Closed

Address v0.52 followup regarding cachedValue in scheduleIBCUpgrade #7625

Open

chore: address comments from pr

6ad08d9

damiannolan merged commit 5cf7d7e into marko/gomod_change Dec 6, 2024
67 checks passed

damiannolan deleted the damian/runtime-env branch December 6, 2024 09:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add runtime Env to icacontroller and fix v0.52 e2e tests #7587

feat: add runtime Env to icacontroller and fix v0.52 e2e tests #7587

damiannolan commented Nov 21, 2024 •

edited

Loading

damiannolan commented Nov 25, 2024

damiannolan commented Dec 2, 2024 •

edited

Loading

damiannolan commented Dec 2, 2024 •

edited

Loading

damiannolan commented Dec 4, 2024 •

edited

Loading

damiannolan Dec 5, 2024

gjermundgaraba Dec 5, 2024

damiannolan Dec 5, 2024

gjermundgaraba Dec 5, 2024

damiannolan Dec 5, 2024

gjermundgaraba Dec 5, 2024

damiannolan Dec 6, 2024

gjermundgaraba left a comment

gjermundgaraba Dec 5, 2024

damiannolan Dec 6, 2024

gjermundgaraba Dec 5, 2024

damiannolan Dec 6, 2024

gjermundgaraba Dec 5, 2024

gjermundgaraba Dec 5, 2024

DimitrisJim Dec 6, 2024

gjermundgaraba Dec 5, 2024

DimitrisJim left a comment

sonarqubecloud bot commented Dec 6, 2024

		// TODO(damian): revert this extra err info
		return nil, errorsmod.Wrapf(ibcerrors.ErrUnpackAny, "cannot unpack Any into ClientState %T, typeURL: %s, cachedValue: %v", protoAny, protoAny.TypeUrl, protoAny.GetCachedValue())

feat: add runtime Env to icacontroller and fix v0.52 e2e tests #7587

feat: add runtime Env to icacontroller and fix v0.52 e2e tests #7587

Conversation

damiannolan commented Nov 21, 2024 • edited Loading

Description

damiannolan commented Nov 25, 2024

damiannolan commented Dec 2, 2024 • edited Loading

damiannolan commented Dec 2, 2024 • edited Loading

damiannolan commented Dec 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gjermundgaraba left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DimitrisJim left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Dec 6, 2024

Quality Gate passed for 'ibc-go'

damiannolan commented Nov 21, 2024 •

edited

Loading

damiannolan commented Dec 2, 2024 •

edited

Loading

damiannolan commented Dec 2, 2024 •

edited

Loading

damiannolan commented Dec 4, 2024 •

edited

Loading