-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proof-of-concept: property based tests replace diff tests #667
Conversation
…e of the way init works
Tagged this as difftest, enhancement and testing. I would like to close this, this is an example of a possible future direction we may take. |
Seems ok to close this. Property based testing using the rapid package, combined with tendermock, would make a super useful testing framework that could replace both diff tests and integration tests |
* Backport commits from main to v3 release branch (cosmos#682) * reorganize channel handshake handler (cosmos#647) * reorganize channel handshake handler split out channel state changes into its own function. * readjust 27-interchain-accounts to not rely on state being set before the application callback * add changelog and migration doc entry * Update modules/core/04-channel/keeper/handshake.go * docs: ICA Overview (cosmos#626) * docs: ica overview * fix: ordering * add spacing * fix: spacing * fix: remove bulletpoints * fix: wording * update go mod for security vulnerabilities (cosmos#655) * update go mod for security vulnerabilities * update package-lock.json * update vue dependency (cosmos#662) * bump glob-parent version in json package (cosmos#663) * build(deps): bump actions/setup-go from 2.1.4 to 2.1.5 (cosmos#656) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 2.1.4 to 2.1.5. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v2.1.4...v2.1.5) --- updated-dependencies: - dependency-name: actions/setup-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: colin axnér <[email protected]> * docs: begin removal of internal "spec" directories (cosmos#634) * begin removal of spec docs within core ibc * remove broken link * Apply suggestions from code review Co-authored-by: Damian Nolan <[email protected]> * remove broken link * remove broken links * Apply suggestions from code review Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Damian Nolan <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> * Modify `OnChanOpenTry` application callback to perform app version negotitation (cosmos#646) * remove NegotiateAppVersion and AppVersion gRPC (cosmos#643) The NegotiateAppVersion callback has been removed from the IBC Application interface. The gRPC AppVersion has been removed. The app version negoitation will be handled by applications by returning the version in OnChanOpenTry. * Modify `OnChanOpenTry` to return application version (cosmos#650) * modify OnChanOpenTry to return negotiated version modify IBCModule interface function OnChanOpenTry to return the negotiated app version. Tests have not been updated * fix ibc_module_test.go tests * fix tests * Apply suggestions from code review * add handshake test case * add CHANGELOG and migration docs * update documentation * fix broken link * fix broken link (cosmos#664) * chore: update make build-docs, add docs build checker (cosmos#667) * update Makefile, add docs build checker * Update .github/workflows/check-docs.yml Co-authored-by: Marko <[email protected]> Co-authored-by: Marko <[email protected]> * register ICA query server, fix panics in params query cli (cosmos#666) Register the controller and host query servers to a chain. Returns an error upon cli params query failure instead of panicing. * update of roadmap with latest release (cosmos#653) * update of roadmap with latest release and changed the way release versions are encoded * fixed typo Co-authored-by: Carlos Rodriguez <[email protected]> * build(deps): bump actions/checkout from 2.3.1 to 2.4.0 (cosmos#672) Bumps [actions/checkout](https://github.com/actions/checkout) from 2.3.1 to 2.4.0. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2.3.1...v2.4.0) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * allow ics20 to connect to middleware (cosmos#675) * allow ics20 to connect to middleware Creates ics4Wrapper which allows middleware applications to only implement SendPacket to connect ics20 as an application in its middleware stack * add changelog and migration doc * fix migration doc spelling * fix: register InterchainAccount as x/auth GenesisAccount (cosmos#676) * adding GenesisAccount interface registration for InterchainAccount impl * updating RegisterInterfaces ica godoc * enable mergify for backports (cosmos#678) Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Sean King <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Damian Nolan <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Marko <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> * fixes evmos tests * cleanup * changelog and cleanup * pr suggestions. bump chainid Co-authored-by: colin axnér <[email protected]> Co-authored-by: Sean King <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Damian Nolan <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Marko <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Federico Kunze Küllmer <[email protected]>
## Note, permanently a draft PR. For education purpose only, for now.
Property-Based Testing for Interchain Security normal operation
What is this?
Some code which is an example of how you can test very high level properties of Interchain Security using property based testing, and in particular the rapid golang library for property based testing.
Why is this interesting?
It's a concrete demonstration of using PBT in the interchain security context to achieve a high coverage in a very small amount of code. Additionally, the system is viewed very much as a black box so this test should not need to change hardly at all.
Where can we learn about property based testing?
See Tutorial.md for a tutorial and links to more resources about the property based testing idea and how to use the rapid library. See Method.md to understand diff-testing, which is not the same as PBT, but gives context. Also, Method.md DOES contain a section comparing diff testing and property based testing.
What does this directory contain?
One property based test using the rapid library in pbt_test.go. The test relies on some state setup, which is all done in setup/setup.go. There is a normal unit test for the setup itself in setup/setup_test.go.
The property based test takes 1 provider chain and 1 consumer chain which are connected and which share the same validator set. It simulates an ibc channel between them. Then, it does random actions against the system to check that Validator Set Replication property holds.
The actions are
The property(s) are
What can we do with this?
Whatever you like. If you find it interesting and useful you can bring it into the main branch. Also, you may consider moving towards a PBT based approach in future, as opposed to a diff testing one. See Method.md for more info.