Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate ES Query tests to deployment agnostic solution #183395

Closed
mgiota opened this issue May 14, 2024 · 2 comments · Fixed by #195715
Closed

Migrate ES Query tests to deployment agnostic solution #183395

mgiota opened this issue May 14, 2024 · 2 comments · Fixed by #195715
Assignees
Labels
Team:obs-ux-management Observability Management User Experience Team

Comments

@mgiota
Copy link
Contributor

mgiota commented May 14, 2024

Part of #183263

🍒 Summary

As part of this #187924 we introduced a way to write deployment agnostic tests using the deployment agnostic framework from Appex QA team. We want to migrate the existing metric threshold rule tests to the new agnostic framework.

Existing tests

Test Type ESS Serverless Notes
ES query rule type Alerting ESS: x-pack/test_serverless/api_integration/test_suites/observability/es_query_rule

New folder structure

How to migrate

In this PR oblt.stateful.config.ts and oblt.serverless.config.ts were added, so no need to create any config file.

Here's what you have to do:

  • Move ES query rule tests from x-pack/test_serverless/api_integration/test_suites/observability/es_query_rule within x-pack/test/api_integration/deployment_agnostic/apis/observability/alerting folder
  • Add the ES query rule test in this file
    loadTestFile(require.resolve('./es_query_rule');
  • Delete the ES query rule test from the old location
  • Use the DeploymentAgnosticFtrProviderContext
  • use a combination of supertestWithoutAuth and samlAuth to generate an API key for user roles and make API calls
const samlAuth = getService('samlAuth');
const supertestWithoutAuth = getService('supertestWithoutAuth');
  • Use the config service to decide on the expected consumer
const config = getService('config');
const isServerless = config.get('serverless');
const expectedConsumer = isServerless ? 'observability' : 'logs'; // Need to verify if the consumer on stateful is correct

How to run

To run serverless

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="ElasticSearch query rule"

To run stateful

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="ElasticSearch query rule"
@maryam-saeidi
Copy link
Member

Moved to backlog until the related Meta ticket is prioritized.

@jasonrhodes
Copy link
Member

I think we need to update these instructions so they are up to date with the latest on deployment agnostic testing before this issue is "Ready", but then we need to pull this in ASAP.

@mgiota mgiota self-assigned this Oct 10, 2024
mgiota added a commit that referenced this issue Oct 24, 2024
…5715)

Fixes #183395

This PR migrates the ES query rule tests to the [deployment agnostic
framework](https://github.com/elastic/kibana/tree/main/x-pack/test/api_integration/deployment_agnostic)

### TODO

- [x] Migrate ES rule tests into the deployment agnostic solution
- [ ] Test in MKI before merging

### How to run tests locally

To run serverless
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="ElasticSearch query rule"
```

To run stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="ElasticSearch query rule"
```

### How to run tests on MKI

According to this
[discussion](elastic/observability-dev#3519 (comment)),
we should test in MKI environment before merging. For details on how to
run in MKI, see [this section of the
document](https://docs.google.com/document/d/1tiax7xoDYwFXYZjRTgVKkVMjN-SQzBWk4yn1JY6Z5UY/edit#heading=h.ece2z8p74izh)
and [this
readme](https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki).

Co-authored-by: Elastic Machine <[email protected]>
Co-authored-by: Maryam Saeidi <[email protected]>
mgiota added a commit to mgiota/kibana that referenced this issue Nov 5, 2024
…stic#195715)

Fixes elastic#183395

This PR migrates the ES query rule tests to the [deployment agnostic
framework](https://github.com/elastic/kibana/tree/main/x-pack/test/api_integration/deployment_agnostic)

### TODO

- [x] Migrate ES rule tests into the deployment agnostic solution
- [ ] Test in MKI before merging

### How to run tests locally

To run serverless
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="ElasticSearch query rule"
```

To run stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="ElasticSearch query rule"
```

### How to run tests on MKI

According to this
[discussion](elastic/observability-dev#3519 (comment)),
we should test in MKI environment before merging. For details on how to
run in MKI, see [this section of the
document](https://docs.google.com/document/d/1tiax7xoDYwFXYZjRTgVKkVMjN-SQzBWk4yn1JY6Z5UY/edit#heading=h.ece2z8p74izh)
and [this
readme](https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki).

Co-authored-by: Elastic Machine <[email protected]>
Co-authored-by: Maryam Saeidi <[email protected]>
(cherry picked from commit 512bfcc)

# Conflicts:
#	x-pack/test/api_integration/deployment_agnostic/apis/observability/alerting/index.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:obs-ux-management Observability Management User Experience Team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants