Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[serverless] Response Ops ESS test skips #165145

Closed
1 task
Ikuni17 opened this issue Aug 29, 2023 · 1 comment · Fixed by #165514
Closed
1 task

[serverless] Response Ops ESS test skips #165145

Ikuni17 opened this issue Aug 29, 2023 · 1 comment · Fixed by #165514
Assignees
@ymao1
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@Ikuni17
Copy link
Contributor

Ikuni17 commented Aug 29, 2023

Tests which were failing and skipped in #162673 due to swapping from ES snapshot to serverless.

  • x-pack/test_serverless/api_integration/test_suites/common/alerting/rules.ts
@Ikuni17 Ikuni17 added bug Fixes for quality problems that affect the customer experience failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) and removed bug Fixes for quality problems that affect the customer experience labels Aug 29, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Aug 29, 2023
@Ikuni17 Ikuni17 mentioned this issue Aug 29, 2023
Merged
2 tasks
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Aug 29, 2023
Ikuni17 added a commit that referenced this issue Aug 30, 2023
@Ikuni17 @dmlemeshko
@mistic @kibanamachine @patrykkopycinski
ESS support for FTR serverless tests. SSL support in kbn/es. kbn/es D…
06ebc31 
…X improvements. (#162673)

Closes #162593
Closes #163939 
Closes #162625

The original intention of this PR was to add FTR support for ESS.
However the scope increased as that also required adding SSL support due
to tests failing from disabled `security` and no authentication.
Additionally, after using serverless in `kbn/es` extensively for this,
there was a bit of friction in regards to DX.

## Summary
- Switch `x-pack/test_serverless` FTR to use ES serverless instead of
(stateful) snapshot
- Adds SSL support to Docker and Serverless in `kbn/es`
- Adds `port` option override
- Adds `teardown` option to kill running nodes if the process exits
without shutdown
- Adds `kill` option to kill running nodes on startup if detected
- Adds `--esFrom serverless` to FTR CLI
- Adds `files` option to mount extra files into containers
- For serverless, automatically attach to first node with `docker logs
-f es01` on startup for better DX.
- Added `background` flag to not attach `logs`.
- Adds graceful shutdown for ESS cluster
- Separate `docker pull` from `run` for better logging, ensures latest
image and stops multiple pulls of the same image occurring in parallel
- Align (most) default settings for ES serverless with `gradlew`
[settings](https://github.com/elastic/elasticsearch-serverless/blob/main/serverless-build-tools/src/main/kotlin/elasticsearch.serverless-run.gradle.kts#L8)
- Fixes Docker bind mount permissions in CI
- Fixes issue where `esFrom` would default to `snapshot` and override
FTR config settings.

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

## Related Issues for Skipped Tests
Security Threat Hunting: #165135
Observability: #165138
Response Ops: #165145

---------

Co-authored-by: Dzmitry Lemechko <[email protected]>
Co-authored-by: Tiago Costa <[email protected]>
Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Patryk Kopycinski <[email protected]>
eokoneyo pushed a commit to eokoneyo/kibana that referenced this issue Aug 31, 2023
@Ikuni17 @dmlemeshko
@mistic @kibanamachine @patrykkopycinski @eokoneyo
ESS support for FTR serverless tests. SSL support in kbn/es. kbn/es D…
18b3ba1 
…X improvements. (elastic#162673)

Closes elastic#162593
Closes elastic#163939 
Closes elastic#162625

The original intention of this PR was to add FTR support for ESS.
However the scope increased as that also required adding SSL support due
to tests failing from disabled `security` and no authentication.
Additionally, after using serverless in `kbn/es` extensively for this,
there was a bit of friction in regards to DX.

## Summary
- Switch `x-pack/test_serverless` FTR to use ES serverless instead of
(stateful) snapshot
- Adds SSL support to Docker and Serverless in `kbn/es`
- Adds `port` option override
- Adds `teardown` option to kill running nodes if the process exits
without shutdown
- Adds `kill` option to kill running nodes on startup if detected
- Adds `--esFrom serverless` to FTR CLI
- Adds `files` option to mount extra files into containers
- For serverless, automatically attach to first node with `docker logs
-f es01` on startup for better DX.
- Added `background` flag to not attach `logs`.
- Adds graceful shutdown for ESS cluster
- Separate `docker pull` from `run` for better logging, ensures latest
image and stops multiple pulls of the same image occurring in parallel
- Align (most) default settings for ES serverless with `gradlew`
[settings](https://github.com/elastic/elasticsearch-serverless/blob/main/serverless-build-tools/src/main/kotlin/elasticsearch.serverless-run.gradle.kts#L8)
- Fixes Docker bind mount permissions in CI
- Fixes issue where `esFrom` would default to `snapshot` and override
FTR config settings.

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

## Related Issues for Skipped Tests
Security Threat Hunting: elastic#165135
Observability: elastic#165138
Response Ops: elastic#165145

---------

Co-authored-by: Dzmitry Lemechko <[email protected]>
Co-authored-by: Tiago Costa <[email protected]>
Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Patryk Kopycinski <[email protected]>
@mikecote mikecote moved this from Awaiting Triage to Todo in AppEx: ResponseOps - Execution & Connectors Sep 1, 2023
@ymao1 ymao1 self-assigned this Sep 1, 2023
@ymao1 ymao1 moved this from Todo to In Progress in AppEx: ResponseOps - Execution & Connectors Sep 1, 2023
@ymao1 ymao1 mentioned this issue Sep 1, 2023
Merged
@ymao1 ymao1 moved this from In Progress to In Review in AppEx: ResponseOps - Execution & Connectors Sep 1, 2023
ymao1 added a commit that referenced this issue Sep 5, 2023
@ymao1 @kibanamachine
[Response Ops] Unskip serverless alerting rule tests (#165514)
74f3ba2 
Resolves #165145

## Summary

Serverless alerting tests are now working, likely after the PR to
support datastreams in AAD, so tests can be unskipped.

Can see the test being run as part of the [serverless security
tests](https://buildkite.com/elastic/kibana-pull-request/builds/155830#018a5221-26e0-4c9c-aada-a1f0bb53a22f/279-908),
the [serverless search
tests](https://buildkite.com/elastic/kibana-pull-request/builds/155830#018a5221-26e0-4c9c-aada-a1f0bb53a22f/279-908)
and the [serverless observability
tests](https://buildkite.com/elastic/kibana-pull-request/builds/155830#018a5264-9b1d-4bf3-9b91-13ea86d86737/279-940)

---------

Co-authored-by: kibanamachine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ymao1 ymao1

Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
No open projects
AppEx: ResponseOps - Execution & Connectors
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Response Ops] Unskip serverless alerting rule tests ymao1/kibana
3 participants
@ymao1 @Ikuni17 @elasticmachine