To increase confidence in the correctness of the Interchain Security code we consider various testing approaches.
Unit tests are useful for simple standalone functionality, and CRUD operations. Unit tests should use golang's standard testing package, and be defined in files formatted as <file being tested>_test.go in the same directory as the file being tested, following standard conventions.
Mocked external keepers (implemented with gomock) are available for testing code that briefly interacts with external modules, but still only a single function/method relevant to ccv, and a single chain. Ie. do not use mocked external keepers to test the integration of the ccv module with external modules, or integration between consumer and provider.
e2e-tests utilize the IBC Testing Package, and test functionality that is wider in scope than a unit test, but still able to be validated in-memory. Ie. code where advancing blocks would be useful, simulated handshakes, simulated packet relays, etc.
To run e2e tests against your own consumer/provider implementations, use instance_test.go as an example. All you'll need to do is make sure your applications implement the necessary interfaces defined in interfaces.go, pattern match specific_setup.go, then pass in the appropriate types and parameters to the suite, as is done in instance_test.go for the dummy provider/consumer implementations.
Similar to e2e tests, but they compare the system state to an expected state generated from a model implementation.
Integration tests run true consumer and provider chain binaries within a docker container and are relevant to the highest level of functionality. Integration tests use queries/transactions invoked from CLI to drive and validate the code.
Tests can be run using make:
# run unit, e2e, diff, and integration tests
make test
# run unit and e2e tests - prefer this for local development
make test-short
# run difference tests
make test-diff
# run integration tests
make test-integration
# equivalent to make test with caching disabled
make test-no-cacheAlternatively you can run tests using go test:
# run all unit, e2e, and diff tests using go
go test ./...
# run all unit, e2e, and diff tests with verbose output
go test -v ./..
# run all unit, e2e, and diff tests with coverage stats
go test -coverpkg=./x/... -coverprofile=coverage.out ./...
# run a single unit test
go test -run <unit-test-name> path/to/package
# example: run a single unit test
go test -run TestSlashAcks ./x/ccv/provider/keeper
# run a single e2e test
go test -run <test-suite-name>/<test-name> ./...
# example: run a single e2e test
go test -run TestProviderTestSuite/TestPacketRoundtrip ./...
# run all integration tests
go run ./tests/integration/...
# run all integration tests with a local cosmos sdk
go run ./tests/integration/... --local-sdk-path "/Users/bob/Documents/cosmos-sdk/"
# run golang native fuzz tests (https://go.dev/doc/tutorial/fuzz)
go test -fuzz=<regex-to-match-test-name>
# run verbose integration tests
go run ./tests/integration/... --local-sdk-path "/Users/bob/Documents/cosmos-sdk/" --verboseSeveral analyzers are used on the code including CodeQL, SonarCloud, golangci-lint and gosec. Some of these are run on github when committing to PRs ect, but some tools are also applicable locally, and are built into golang.
# gofmt to format and simplify code (https://pkg.go.dev/cmd/gofmt)
gofmt -w -s -e .
# go vet to search for suspicious code (https://pkg.go.dev/cmd/vet)
go vet ./...Some useful tools are included in the repository using pre-commit. pre-commit lets you run developer tools either on every git commit, or manually with pre-commit run --all-files. See the config for details. In this repo the hooks are not installed to git, as that can be cumbersome, but it is still possible to benefit from them.
## Prerequisites
# pre-commit
brew install pre-commit
# goimports (https://pkg.go.dev/golang.org/x/tools/cmd/goimports)
go install golang.org/x/tools/cmd/goimports@latest
# gocyclo (https://github.com/fzipp/gocyclo)
go install github.com/fzipp/gocyclo/cmd/gocyclo@latest
# go-critic https://github.com/go-critic/go-critic
go install github.com/go-critic/go-critic/cmd/gocritic@latest
## Run the tools
pre-commit run --all-filesIf using VSCode, see vscode-go/wiki/debugging to debug unit tests or go binaries.
More instructions will be added soon, in time for the testnet.