Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][API testing][Rule Management] Move and restructures remaining rule managements trail-tests #172173

Merged
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions .buildkite/ftr_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -230,8 +230,6 @@ enabled:
- x-pack/test/cloud_security_posture_api/config.ts
- x-pack/test/dataset_quality_api_integration/basic/config.ts
- x-pack/test/detection_engine_api_integration/basic/config.ts
- x-pack/test/detection_engine_api_integration/security_and_spaces/group1/config.ts
- x-pack/test/detection_engine_api_integration/security_and_spaces/group10/config.ts
- x-pack/test/disable_ems/config.ts
- x-pack/test/encrypted_saved_objects_api_integration/config.ts
- x-pack/test/examples/config.ts
Expand Down Expand Up @@ -495,5 +493,17 @@ enabled:
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/exception_lists_items/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/lists_items/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/lists_items/configs/ess.config.ts


- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_delete/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_delete/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_update/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_update/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_patch/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_patch/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_import_export/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_import_export/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_management/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_management/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_read/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_read/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_bulk_actions/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_bulk_actions/configs/serverless.config.ts
97 changes: 86 additions & 11 deletions .buildkite/pipelines/security_solution/api_integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ steps:
key: exception_operators_date_numeric_types:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '*'
Expand All @@ -31,7 +31,7 @@ steps:
automatic:
- exit_status: '*'
limit: 2

- label: Running exception_operators_ips:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh exception_operators_ips:qa:serverless
key: exception_operators_ips:qa:serverless
Expand All @@ -53,7 +53,6 @@ steps:
automatic:
- exit_status: '1'
limit: 2


- label: Running exception_operators_text:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh exception_operators_text:qa:serverless
Expand All @@ -76,7 +75,7 @@ steps:
automatic:
- exit_status: '1'
limit: 2

- label: Running actions:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh actions:qa:serverless
key: actions:qa:serverless
Expand All @@ -97,7 +96,7 @@ steps:
retry:
automatic:
- exit_status: '1'
limit: 2
limit: 2

- label: Running prebuilt_rules_management:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh prebuilt_rules_management:qa:serverless
Expand All @@ -119,7 +118,7 @@ steps:
retry:
automatic:
- exit_status: '1'
limit: 2
limit: 2

- label: Running prebuilt_rules_large_prebuilt_rules_package:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh prebuilt_rules_large_prebuilt_rules_package:qa:serverless
Expand All @@ -130,7 +129,7 @@ steps:
retry:
automatic:
- exit_status: '1'
limit: 2
limit: 2

- label: Running prebuilt_rules_update_prebuilt_rules_package:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh prebuilt_rules_update_prebuilt_rules_package:qa:serverless
Expand All @@ -152,8 +151,8 @@ steps:
retry:
automatic:
- exit_status: '1'
limit: 2
limit: 2

- label: Running user_roles:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh user_roles:qa:serverless
key: user_roles:qa:serverless
Expand All @@ -163,7 +162,7 @@ steps:
retry:
automatic:
- exit_status: '1'
limit: 2
limit: 2

- label: Running telemetry:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh telemetry:qa:serverless
Expand All @@ -175,4 +174,80 @@ steps:
automatic:
- exit_status: '1'
limit: 2


- label: Running rule_delete:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh rule_delete:qa:serverless
key: rule_delete:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '1'
limit: 2

- label: Running rule_update:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh rule_update:qa:serverless
key: rule_update:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '1'
limit: 2

- label: Running rule_patch:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh rule_patch:qa:serverless
key: rule_patch:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '1'
limit: 2

- label: Running rule_import_export:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh rule_import_export:qa:serverless
key: rule_import_export:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '1'
limit: 2

- label: Running rule_management:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh rule_management:qa:serverless
key: rule_management:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '1'
limit: 2

- label: Running rule_bulk_actions:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh rule_bulk_actions:qa:serverless
key: rule_bulk_actions:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '1'
limit: 2

- label: Running rule_read:qa:serverless
command: .buildkite/scripts/pipelines/security_solution_quality_gate/api-integration-tests.sh rule_read:qa:serverless
key: rule_read:qa:serverless
agents:
queue: n2-4-spot
timeout_in_minutes: 120
retry:
automatic:
- exit_status: '1'
limit: 2
20 changes: 19 additions & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -1329,6 +1329,14 @@ x-pack/test/security_solution_cypress/cypress/tasks/expandable_flyout @elastic/
/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules @elastic/security-detection-rule-management
/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/rule_management @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/prebuilt_rules @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_delete @elastic/security-detection-rule-management
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_update @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_patch @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_import_export @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_management @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_bulk_actions @elastic/security-detection-rule-management
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_read @elastic/security-detection-rule-management
Comment on lines +1347 to +1353
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big fan of the folder structure proposed in this PR.

I think these are too many folders to track them all separately in CODEOWNERS. It would make it much easier for both code navigation and CODEOWNERS if we grouped these test groups into higher-level folders reflecting our subdomains. In this case, all these folders could be under a rule_management folder so we'd only need to add a single line to this file here:

/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_management @elastic/security-detection-rule-management

For the sake of having less pain I'd suggest to merge this PR as is and restructure these folders in a follow-up one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copying from other comment:

@banderror let's definitely chat! I plan to address the comments that don't deal with restructuring first, and then open a PR dedicated to any structure updates. For the ownership on the endpoints we largely followed what we saw in the folders structure in the code itself (these routes are under rule_management, but it definitely felt a bit confusing.

I'll find a time to sync!

/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation/create_rules_bulk.ts @elastic/security-detection-rule-management
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we add a single file to CODEOWNERS? We shouldn't target individual files in CODEOWNERS because this can get out of sync with actual files pretty quickly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copying from other comment:

@banderror let's definitely chat! I plan to address the comments that don't deal with restructuring first, and then open a PR dedicated to any structure updates. For the ownership on the endpoints we largely followed what we saw in the folders structure in the code itself (these routes are under rule_management, but it definitely felt a bit confusing.

I'll find a time to sync!


/x-pack/plugins/security_solution/public/common/components/health_truncate_text @elastic/security-detection-rule-management
/x-pack/plugins/security_solution/public/common/components/links_to_docs @elastic/security-detection-rule-management
Expand Down Expand Up @@ -1393,14 +1401,24 @@ x-pack/test/security_solution_cypress/cypress/tasks/expandable_flyout @elastic/
/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists @elastic/security-detection-engine
/x-pack/test/security_solution_cypress/cypress/e2e/exceptions @elastic/security-detection-engine
/x-pack/test/security_solution_cypress/cypress/e2e/overview @elastic/security-detection-engine

/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation/create_new_terms @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation/create_rules @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation/preview_rules @elastic/security-detection-engine
Comment on lines +1414 to +1416
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't target individual files in CODEOWNERS because this can get out of sync with actual files pretty quickly.

Copy link
Contributor

@yctercero yctercero Dec 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copying from other comment:

@banderror let's definitely chat! I plan to address the comments that don't deal with restructuring first, and then open a PR dedicated to any structure updates. For the ownership on the endpoints we largely followed what we saw in the folders structure in the code itself (these routes are under rule_management, but it definitely felt a bit confusing.

I'll find a time to sync!

/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/user_roles @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/detection_engine @elastic/security-detection-engine
/x-pack/plugins/security_solution/server/lib/detection_engine/scripts/roles_users @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_update/update_rules.ts @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_update/update_rules_ess.ts @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_patch/patch_rules.ts @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_patch/patch_rules_ess.ts @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_delete/delete_rules.ts @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_delete/delete_rules_ess.ts @elastic/security-detection-engine
Comment on lines +1423 to +1428
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here regarding targeting individual files.

Also, I'd like to chat with you @yctercero about which team should own which rules CRUD tests. I don't understand the proposed separation between our two teams, and I have some thoughts about how we should do it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@banderror let's definitely chat! I plan to address the comments that don't deal with restructuring first, and then open a PR dedicated to any structure updates. For the ownership on the endpoints we largely followed what we saw in the folders structure in the code itself (these routes are under rule_management, but it definitely felt a bit confusing.

I'll find a time to sync!



## Security Threat Intelligence - Under Security Platform
/x-pack/plugins/security_solution/public/common/components/threat_match @elastic/security-detection-engine
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ import {
deleteAllRules,
getSimpleRule,
installPrebuiltRulesAndTimelines,
createNonSecurityRule,
} from '../../utils';
import { createNonSecurityRule } from '../../utils/create_non_security_rule';

// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext): void => {
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

Loading