Add integration test coverage for SO migrations against serverless ES #164959
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s-against-serverless
pgayvallet
added
Team:Core
…s-against-serverless
pgayvallet
changed the title
pgayvallet
changed the title
|
Pinging @elastic/kibana-core (Team:Core) |
jloleysens
approved these changes
Sep 1, 2023
There was a problem hiding this comment.
Overall approach makes sense to me.
I am a little bit concerned about maintainability when we start to use multiple createXTest test factories, but the alternative I can think of is to maintain two sets of tests: one for serverless and one for stateful.
Overall, for now, I think this is a pragmatic compromise.
…s-against-serverless
pgayvallet
force-pushed
the
kbn-xxx-action-tests-against-serverless
branch
from
41d54c0
to
a5aefd4
Compare
…s-against-serverless
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Public APIs missing exports
History
To update your PR or re-run it, just comment with: |
rudolf
added a commit
that referenced
this pull request
Oct 27, 2023
…ness (#170023) Attempt at understanding #166190 I first thought the flakiness could be a regression in Elasticsearch, but it turns out because I can only reproduce 90% of the time I got "lucky" and found a previous ES commit where tests passed 3 times in a row. On further investigation I was able to reproduce failures on commits as old as July (before we even saw these failures) so ES is unlikely to be the root cause 👎 Looking at our flaky test failure logs I found the first occurence happened on 5 September on the branch for #164959 (before it was merged) and then subsequent failures on other branches and main happened within a few hours after #164959 was merged on 11 September. However, nothing in #164959 should have changed or impacted behaviour. To try to track down the cause this PR is attempting to restorte `actions.test.ts` until before #164959. Serverless tests will continue to use `actions_test_suite.ts` but for classic we'll run the old `actions.test.ts`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Blocked by #162673Add some initial integration test coverage for SO migrations when running against serverless Elasticsearch:
The actions test suite was adapted to skip, when run against serverless, the tests that are not supposed to be run (or passing) in that environment