Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/oasis-test-runner/cmd: Limit scenario name regex matching #3103

Merged
merged 1 commit into from
Jul 14, 2020

Conversation

tjanez
Copy link
Member

@tjanez tjanez commented Jul 13, 2020

Prevent oasis-test-runner to match too many scenarios for a given scenario name regex by ensuring the given scenario name regex matches the whole scenario name.

For example, previously, using

oasis-test-runner --test a [... other arguments ...]

would match and run all the following scenarios:

e2e/runtime/runtime-dynamic
e2e/runtime/keymanager-replication
e2e/gas-fees/staking
e2e/runtime/byzantine/executor-straggler
e2e/runtime/gas-fees/runtimes
e2e/node-upgrade
e2e/runtime/keymanager-upgrade
e2e/runtime/runtime-upgrade
e2e/gas-fees/staking-dump-restore
e2e/early-query
e2e/stake-cli
e2e/runtime/byzantine/merge-honest
e2e/runtime/storage-sync
remote-signer/basic
e2e/runtime/byzantine/merge-wrong
e2e/runtime/byzantine/merge-straggler
e2e/runtime/byzantine/executor-honest
e2e/node-upgrade-cancel
e2e/runtime/keymanager-restart
e2e/runtime/halt-restore
e2e/runtime/late-start
e2e/runtime/byzantine/executor-wrong

Now, it doesn't match any scenario.

If one wants to match something like before, he needs to explicitly add wildcards, e.g.:

oasis-test-runner --test .*a.* [... other arguments ...]

@tjanez tjanez added the c:testing Category: testing label Jul 13, 2020
@tjanez tjanez self-assigned this Jul 13, 2020
@tjanez tjanez force-pushed the tjanez/oasis-test-runner-limit-regex branch from ea509b0 to d13457b Compare July 13, 2020 10:24
@codecov
Copy link

codecov bot commented Jul 13, 2020

Codecov Report

Merging #3103 into master will increase coverage by 0.07%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3103      +/-   ##
==========================================
+ Coverage   68.30%   68.38%   +0.07%     
==========================================
  Files         372      372              
  Lines       36896    36896              
==========================================
+ Hits        25201    25230      +29     
+ Misses       8436     8406      -30     
- Partials     3259     3260       +1     
Impacted Files Coverage Δ
go/consensus/tendermint/apps/roothash/query.go 68.42% <0.00%> (-10.53%) ⬇️
go/runtime/client/watcher.go 73.84% <0.00%> (-4.62%) ⬇️
.../consensus/tendermint/apps/roothash/state/state.go 68.62% <0.00%> (-3.93%) ⬇️
go/consensus/tendermint/roothash/roothash.go 66.00% <0.00%> (-1.99%) ⬇️
...consensus/tendermint/apps/registry/transactions.go 55.49% <0.00%> (-1.61%) ⬇️
go/worker/common/committee/group.go 80.88% <0.00%> (-0.69%) ⬇️
go/consensus/tendermint/abci/state.go 68.02% <0.00%> (-0.69%) ⬇️
.../consensus/tendermint/apps/registry/state/state.go 58.61% <0.00%> (-0.56%) ⬇️
go/consensus/tendermint/abci/mux.go 68.43% <0.00%> (-0.47%) ⬇️
go/consensus/tendermint/tendermint.go 64.91% <0.00%> (-0.22%) ⬇️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db0c32b...fecdd17. Read the comment docs.

Prevent oasis-test-runner to match too many scenarios for a given
scenario name regex by ensuring the given scenario name regex matches
the whole scenario name.
@tjanez tjanez force-pushed the tjanez/oasis-test-runner-limit-regex branch from d13457b to fecdd17 Compare July 14, 2020 07:58
@tjanez tjanez merged commit 0466b12 into master Jul 14, 2020
@tjanez tjanez deleted the tjanez/oasis-test-runner-limit-regex branch July 14, 2020 08:14
ptrus pushed a commit that referenced this pull request Jul 14, 2020
…limit-regex

go/oasis-test-runner/cmd: Limit scenario name regex matching
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c:testing Category: testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants