Skip to content

Commit

Permalink
Common folder for tests (#325)
Browse files Browse the repository at this point in the history
* move to dir

* other stuff

* revert one change

* Update Dockerfile

* Squashed commit , with resolutions, of the following:

commit 7e639da
Author: Daniel T <[email protected]>
Date:   Tue Sep 6 11:46:08 2022 +0100

    Difference testing for 'core' protocol using model generated (heuristic) traces (#315)

    * blanket add difftest

    * (assum + trace pass) upstream sync setup

    * (full pass) shuffle directories

    * (all pass) minor tidyup simibc interface

    * (all pass) removes extraneous updateClients, deliverAcks

    * (pass) del extra UpdateClient in framework deliverAcks

    * (pass) tweak docstrings

    * (pass) remove chains map from framework

    * (pass) rename, doc simibc

    * (regression) tweaks

    * (pass) tweaks

    * (all pass) bump max gass

    * Track traces

    * Bumps ci configs

    * Try fix gosec

    * Try fix sonar

    * Try fix docker go build for 1.18

    * README

    * Move diff artifacts to one dir

    * Update README

    * Update git LFS

    * Update sonar properties

    * Update README

    * Rename NetworkLink -> OrderedLink

    * Better docs main.ts

    * Docstring matchState

    * Removes some hardcoded nums in core_test.go

    * Adds couple of comments to core_test.go

    * Docstring for Consequence (.ts)

    * Update core package.json

    * Ensure typechecks enabled

    * Prettier on model.ts

    * DEL .editorconfig for ts model

    * del eslint

    * Bump sonar

    * Minor tidyup Traces struct go

    * Del unused field in properties.ts

    * new semantics: model.ts::expired -> willBeProcessedByStakingModule

    * Better comment DeliverPackets

    * Better comments for model class fields

    * Improve model staking class docstrings

    * Improve model staking class docstrings

    * Fix more_one_third_val_power_change event

    * Update testutil/simibc/ordered_link.go comment

    Co-authored-by: Shawn Marshall-Spitzbart <[email protected]>

    * Adds event.receive_slash_request_unbonded

    * Adds event.receive_slash_request_unbonded

    * Better docstring BlockHistory

    * Adds 4 new events to constants

    * Model endblock method split

    * Couple more comments model.ts

    * Comment model.ts

    * New traces

    * DEL traces

    * CP (debug)

    * (debug) offset adjustment

    * tentative, fix regression (debug)

    * kill prints

    * (tentative) property fix

    * Fixes BBCVP power provider

    * New traces

    * Del 2x prints

    * Comment property perf

    * rn power -> consumerPower in model

    * rn power -> consumerPower in driver

    * Removes outdated comment

    Co-authored-by: Daniel <[email protected]>
    Co-authored-by: Jehan <[email protected]>
    Co-authored-by: Shawn Marshall-Spitzbart <[email protected]>

* DEL extra files

Co-authored-by: Daniel <[email protected]>
Co-authored-by: Daniel T <[email protected]>
  • Loading branch information
3 people authored Sep 6, 2022
1 parent 7e639da commit d448db8
Show file tree
Hide file tree
Showing 49 changed files with 10 additions and 12 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/automated-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ jobs:
run: go test ./...

- name: Integration tests
run: go run ./integration-tests/...
run: go run ./tests/integration/...
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ COPY --from=is-builder /go/bin/interchain-security-cd /usr/local/bin/interchain-


# Copy in the shell scripts that run the testnet
ADD ./integration-tests/testnet-scripts /testnet-scripts
ADD ./tests/integration/testnet-scripts /testnet-scripts

# Copy in the hermes config
ADD ./integration-tests/testnet-scripts/hermes-config.toml /root/.hermes/config.toml
ADD ./tests/integration/testnet-scripts/hermes-config.toml /root/.hermes/config.toml
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,15 +53,15 @@ Unit tests are useful for simple standalone functionality, and CRUD operations.

### End to End (e2e) Tests

[e2e-tests](./e2e-tests/) utilize the [IBC Testing Package](https://github.com/cosmos/ibc-go/tree/main/testing), and test functionality that is wider in scope than a unit test, but still able to be validated in-memory. Ie. code where advancing blocks would be useful, simulated handshakes, simulated packet relays, etc.
[e2e-tests](./tests/e2e/) utilize the [IBC Testing Package](https://github.com/cosmos/ibc-go/tree/main/testing), and test functionality that is wider in scope than a unit test, but still able to be validated in-memory. Ie. code where advancing blocks would be useful, simulated handshakes, simulated packet relays, etc.

### Differential Tests (WIP)

Similar to e2e tests, but they compare the system state to an expected state generated from a model implementation.

### Integration Tests

[Integration tests](./integration-tests/) run true consumer and provider chain binaries within a docker container and are relevant to the highest level of functionality. Integration tests use queries/transactions invoked from CLI to drive and validate the code.
[Integration tests](./tests/integration/) run true consumer and provider chain binaries within a docker container and are relevant to the highest level of functionality. Integration tests use queries/transactions invoked from CLI to drive and validate the code.

### Running Tests

Expand All @@ -85,9 +85,9 @@ go test -run <test-suite-name>/<test-name> ./...
# example: run a single e2e test
go test -run TestProviderTestSuite/TestPacketRoundtrip ./...
# run all integration tests
go run ./integration-tests/...
go run ./tests/integration/...
# run all integration tests with a local cosmos sdk
go run ./integration-tests/... --local-sdk-path "/Users/bob/Documents/cosmos-sdk/"
go run ./tests/integration/... --local-sdk-path "/Users/bob/Documents/cosmos-sdk/"
# run golang native fuzz tests (https://go.dev/doc/tutorial/fuzz)
go test -fuzz=<regex-to-match-test-name>
```
Expand Down
3 changes: 0 additions & 3 deletions diff-tests/core/driver/traces.json

This file was deleted.

2 changes: 1 addition & 1 deletion sonar-project.properties
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ sonar.organization=cosmos
# All golang artifacts
sonar.sources=.
# Do not calculate coverage metrics for statements in these files
sonar.exclusions=**/vendor/**,**/*.pb.go,**/*.pb.gw.go,proto,**/*_test.go,integration-tests/**,testutil/**,diff-tests/**
sonar.exclusions=**/vendor/**,**/*.pb.go,**/*.pb.gw.go,proto,**/*_test.go,tests/**,testutil/**
sonar.tests=.
# Run unit and e2e tests, but not integration tests
sonar.test.inclusions=**/*_test.go
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
1 change: 1 addition & 0 deletions tests/difference/core/driver/traces.json

Large diffs are not rendered by default.

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion integration-tests/main.go → tests/integration/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ func (tr TestRun) runStep(step Step, verbose bool) {
}

func (tr TestRun) startDocker() {
scriptStr := "integration-tests/testnet-scripts/start-docker.sh " +
scriptStr := "tests/integration/testnet-scripts/start-docker.sh " +
tr.containerConfig.containerName + " " +
tr.containerConfig.instanceName + " " +
tr.localSdkPath
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 comments on commit d448db8

Please sign in to comment.