tests(smoke): add an option and cli flag to control test exclusion #14135
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses #14127.
This implements an
exclude
list to the smoke test definition that should help to exclude a test from a specific runner or all runners.Example:
export default { id: 'foo', expectations, config, + exclude: ['devtools', 'cli'], };
A test can be excluded from all runners by specifying
exclude: ['*']
.In order to temporarily ignore an exclusion for debugging, Smoke CLI now features a boolean flag
--ignore-exclusions
.@connorjclark I chose to retain the list as a fixed structure instead of exclude being a
boolean
.