ES Serverless support for FTR serverless tests. SSL support in kbn/es. kbn/es DX improvements. #162673
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ikuni17
added
WIP
PhilippeOberti
approved these changes
Aug 30, 2023
e40pud
approved these changes
Aug 30, 2023
x-pack/test/security_solution_cypress/cypress/e2e/data_sources/create_runtime_field.cy.ts
Outdated
Show resolved
Hide resolved
x-pack/test/security_solution_cypress/cypress/e2e/data_sources/sourcerer.cy.ts
Outdated
Show resolved
Hide resolved
6 tasks
…-es-serverless-ftr # Conflicts: # .buildkite/pipelines/pull_request/defend_workflows.yml # x-pack/plugins/security_solution/scripts/run_cypress/parallel.ts
3 tasks
💔 Build FailedFailed CI Steps
Metrics [docs]Public APIs missing comments
Unknown metric groupsAPI count
History
To update your PR or re-run it, just comment with: cc @Ikuni17 |
eokoneyo
pushed a commit
to eokoneyo/kibana
that referenced
this pull request
Aug 31, 2023
…X improvements. (elastic#162673) Closes elastic#162593 Closes elastic#163939 Closes elastic#162625 The original intention of this PR was to add FTR support for ESS. However the scope increased as that also required adding SSL support due to tests failing from disabled `security` and no authentication. Additionally, after using serverless in `kbn/es` extensively for this, there was a bit of friction in regards to DX. ## Summary - Switch `x-pack/test_serverless` FTR to use ES serverless instead of (stateful) snapshot - Adds SSL support to Docker and Serverless in `kbn/es` - Adds `port` option override - Adds `teardown` option to kill running nodes if the process exits without shutdown - Adds `kill` option to kill running nodes on startup if detected - Adds `--esFrom serverless` to FTR CLI - Adds `files` option to mount extra files into containers - For serverless, automatically attach to first node with `docker logs -f es01` on startup for better DX. - Added `background` flag to not attach `logs`. - Adds graceful shutdown for ESS cluster - Separate `docker pull` from `run` for better logging, ensures latest image and stops multiple pulls of the same image occurring in parallel - Align (most) default settings for ES serverless with `gradlew` [settings](https://github.com/elastic/elasticsearch-serverless/blob/main/serverless-build-tools/src/main/kotlin/elasticsearch.serverless-run.gradle.kts#L8) - Fixes Docker bind mount permissions in CI - Fixes issue where `esFrom` would default to `snapshot` and override FTR config settings. ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ## Related Issues for Skipped Tests Security Threat Hunting: elastic#165135 Observability: elastic#165138 Response Ops: elastic#165145 --------- Co-authored-by: Dzmitry Lemechko <[email protected]> Co-authored-by: Tiago Costa <[email protected]> Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Patryk Kopycinski <[email protected]>
banderror
added a commit
that referenced
this pull request
Sep 9, 2023
…rverless (#165966) **Resolves: #164441 ## Summary Skips all Cypress tests owned by @elastic/security-detection-rule-management and @elastic/security-detection-engine teams in Serverless using the new `@skipInServerless` tag. - Adds a new `@skipInServerless` tag - Updates `x-pack/test/security_solution_cypress/cypress/README.md` to explain when to use what tags - Explicitly adds missing tags to all D&R tests - Adds `// TODO:` comments to tests with links to follow-up tickets - Fixes the `x-pack/plugins/security_solution/scripts/run_cypress/parallel.ts` script (see below) Follow-up work: - #161540 - #161539 ## Context > Serverless test failures will soon block PR merge > During the development of the serverless test infrastructure, we decided that serverless tests will only soft-fail. That means you see the test failure in your PR but you're still able to merge. We did that mainly in order to not block delivery of stateful features and bug fixes while serverless tests and pipelines were implemented and stabilized. We now have the major building blocks for the serverless test infrastructure in place and will integrate serverless tests in our regular pipelines. As part of this process, we're skipping failing and flaky serverless tests that came in during the soft-fail phase. Once this is done, a PR with serverless test failures can no longer be merged, similar to how we have it for stateful test failures. > We plan to merge this in the next few days and we'll send out another notification when it's done. ## Fixing `parallel.ts` There were two problems with the `x-pack/plugins/security_solution/scripts/run_cypress/parallel.ts` script we use for running Cypress tests: - The script was broken in #162673 (here on [this line](https://github.com/elastic/kibana/pull/162673/files#diff-9f40ced6d29c4fc2709af881680400293d8ce1bc9ebb07b9138d6d99c83c09c9R67)) - I think it has never supported situations when all tests matching a spec pattern (such as `./cypress/e2e/!(investigations|explore)/**/*.cy.ts`) end up being skipped via Cypress tags (such as `@skipInServerless`) Both the issues are fixed in this PR. Code owners are added for this script in the CODEOWNERS file to prevent breaking this script in future PRs.
pgayvallet
added a commit
that referenced
this pull request
Sep 11, 2023
…#164959) ## Summary ~~Blocked by #162673 Add some initial integration test coverage for SO migrations when running against serverless Elasticsearch: - our migration actions test suite - some of the zdt algo migration suites The actions test suite was adapted to skip, when run against serverless, the tests that are not supposed to be run (or passing) in that environment
Ikuni17
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #162593
Closes #163939
Closes #162625
The original intention of this PR was to add FTR support for ESS. However the scope increased as that also required adding SSL support due to tests failing from disabled
securityand no authentication. Additionally, after using serverless inkbn/esextensively for this, there was a bit of friction in regards to DX.Summary
x-pack/test_serverlessFTR to use ES serverless instead of (stateful) snapshotkbn/esportoption overrideteardownoption to kill running nodes if the process exits without shutdownkilloption to kill running nodes on startup if detected--esFrom serverlessto FTR CLIfilesoption to mount extra files into containersdocker logs -f es01on startup for better DX.backgroundflag to not attachlogs.docker pullfromrunfor better logging, ensures latest image and stops multiple pulls of the same image occurring in parallelgradlewsettingsesFromwould default tosnapshotand override FTR config settings.Checklist
Related Issues for Skipped Tests
Security Threat Hunting: #165135
Observability: #165138
Response Ops: #165145