Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ftr] support filtering tests by es version #123289

Merged
merged 19 commits into from
Jan 19, 2022

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Jan 18, 2022

Add a new method to suites like this.tags(), for defining a valid esVersion range for the suite.

describe('...', function () {
  this.onlyEsVersion('<=7')

  ...
})

This version range is then validated against the --es-version flag passed to the functional_test_runner CLI, the version number extracted from the ES_SNAPSHOT_MANIFEST environment variable, or the Kibana version (in that order).

Any test which doesn't match the es version number will be excluded automatically.

In addition to the ability to skip a whole suite tests can be updated to use the esVersion service, which exposes a helpful method for conditional logic based on version ranges:

const esVersion = getService('esVersion');

//...

es.indices.create({
  ...
  include_type_name: esVersion.matchRange('>=8') ? undefined : includeTypeName
})

To see this in action I'm also working on #123312, which backports this PR and skips all the suites which are failing in 7.17 when running that branch against an 8.0 snapshot.

@spalger spalger added release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v7.16.3 v7.17.0 v8.0.0 v8.1.0 labels Jan 19, 2022
@spalger spalger marked this pull request as ready for review January 19, 2022 01:53
@spalger spalger requested a review from a team as a code owner January 19, 2022 01:53
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@spalger
Copy link
Contributor Author

spalger commented Jan 19, 2022

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/test 180 193 +13

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
kibana 956 957 +1
Unknown metric groups

API count

id before after diff
@kbn/test 203 218 +15

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@spalger spalger merged commit 12e63dd into elastic:main Jan 19, 2022
@spalger spalger deleted the implement/es-version-test-filters branch January 19, 2022 22:46
spalger pushed a commit to spalger/kibana that referenced this pull request Jan 19, 2022
Co-authored-by: Kibana Machine <[email protected]>
(cherry picked from commit 12e63dd)
spalger pushed a commit to spalger/kibana that referenced this pull request Jan 19, 2022
Co-authored-by: Kibana Machine <[email protected]>
(cherry picked from commit 12e63dd)

# Conflicts:
#	packages/kbn-es-archiver/src/cli.ts
#	packages/kbn-test/src/functional_test_runner/cli.ts
spalger pushed a commit to spalger/kibana that referenced this pull request Jan 19, 2022
Co-authored-by: Kibana Machine <[email protected]>
(cherry picked from commit 12e63dd)

# Conflicts:
#	packages/kbn-es-archiver/src/cli.ts
#	packages/kbn-test/src/functional_test_runner/cli.ts
@spalger spalger removed the v7.16.3 label Jan 19, 2022
spalger pushed a commit that referenced this pull request Jan 20, 2022
Co-authored-by: Kibana Machine <[email protected]>
(cherry picked from commit 12e63dd)
spalger pushed a commit that referenced this pull request Jan 20, 2022
* [ftr] support filtering tests by es version (#123289)

Co-authored-by: Kibana Machine <[email protected]>
(cherry picked from commit 12e63dd)

# Conflicts:
#	packages/kbn-es-archiver/src/cli.ts
#	packages/kbn-test/src/functional_test_runner/cli.ts

* fix type error in 7.17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v7.17.0 v8.0.0 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants