Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: X-Pack Detection Engine API Integration Tests.x-pack/test/detection_engine_api_integration/security_and_spaces/tests/create_endpoint_exceptions·ts - detection engine api security and spaces enabled Rule exception operators for endpoints operating system types (os_types) endpoints should filter multiple operating system types (os_type) with multiple filter items for an endpoint #115319

Closed
kibanamachine opened this issue Oct 18, 2021 · 1 comment
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.

Comments

@kibanamachine
Copy link
Contributor

A test failed on a tracked branch

Error: expected [ { os: { name: 'Linux' } },
  { os: { name: 'Macos' } } ] to sort of equal [ { os: { name: 'Macos' } },
  { os: { name: 'Linux' } } ]
    at Assertion.assert (/opt/local-ssd/buildkite/builds/kb-cigroup-6-40122bd897a409d1/elastic/kibana-hourly/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.eql (/opt/local-ssd/buildkite/builds/kb-cigroup-6-40122bd897a409d1/elastic/kibana-hourly/kibana/node_modules/@kbn/expect/expect.js:244:8)
    at Context.<anonymous> (test/detection_engine_api_integration/security_and_spaces/tests/create_endpoint_exceptions.ts:265:27)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (/opt/local-ssd/buildkite/builds/kb-cigroup-6-40122bd897a409d1/elastic/kibana-hourly/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16) {
  actual: '[\n' +
    '  {\n' +
    '    "os": {\n' +
    '      "name": "Linux"\n' +
    '    }\n' +
    '  }\n' +
    '  {\n' +
    '    "os": {\n' +
    '      "name": "Macos"\n' +
    '    }\n' +
    '  }\n' +
    ']',
  expected: '[\n' +
    '  {\n' +
    '    "os": {\n' +
    '      "name": "Macos"\n' +
    '    }\n' +
    '  }\n' +
    '  {\n' +
    '    "os": {\n' +
    '      "name": "Linux"\n' +
    '    }\n' +
    '  }\n' +
    ']',
  showDiff: true
}

First failure: CI Build - 7.x

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 18, 2021
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 18, 2021
@azasypkin azasypkin added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Oct 18, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 18, 2021
@FrankHassanabad FrankHassanabad self-assigned this Oct 20, 2021
FrankHassanabad added a commit that referenced this issue Oct 20, 2021
…arios (#115688)

## Summary

Fixes flakes across tests that have either been skipped or have been a source of flake in the categories of:
* Sorting fixes because Elasticsearch can return hits/arrays back in different orders
* Flat array fixes because Elasticsearch can sometimes return `[]` or `[[]]` in-deterministically in some cases 🤷 , so we just flatten the array out completely and test for `[]` within those tests.
* `waitForSignalsToBePresent` was missing in a test and sometimes we would get an empty array response which would fail CI.

Also I audited other tests for `[[]]` and `waitForSignalsToBePresent` and fixed them where they were present or if the `waitForSignalsToBePresent` count was incorrect. This should give us more stability when the CI is under pressure.

Sorting fixes:
#115554
#115321
#115319
#114581


Flat array fixes:
#89052
#115315
#115308
#115304
#115313
#113418

Missing additional check for "waitForSignalsToBePresent" or incorrect number of signals to wait for fixes:
#115310


### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 20, 2021
…arios (elastic#115688)

## Summary

Fixes flakes across tests that have either been skipped or have been a source of flake in the categories of:
* Sorting fixes because Elasticsearch can return hits/arrays back in different orders
* Flat array fixes because Elasticsearch can sometimes return `[]` or `[[]]` in-deterministically in some cases 🤷 , so we just flatten the array out completely and test for `[]` within those tests.
* `waitForSignalsToBePresent` was missing in a test and sometimes we would get an empty array response which would fail CI.

Also I audited other tests for `[[]]` and `waitForSignalsToBePresent` and fixed them where they were present or if the `waitForSignalsToBePresent` count was incorrect. This should give us more stability when the CI is under pressure.

Sorting fixes:
elastic#115554
elastic#115321
elastic#115319
elastic#114581


Flat array fixes:
elastic#89052
elastic#115315
elastic#115308
elastic#115304
elastic#115313
elastic#113418

Missing additional check for "waitForSignalsToBePresent" or incorrect number of signals to wait for fixes:
elastic#115310


### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine added a commit that referenced this issue Oct 20, 2021
…arios (#115688) (#115699)

## Summary

Fixes flakes across tests that have either been skipped or have been a source of flake in the categories of:
* Sorting fixes because Elasticsearch can return hits/arrays back in different orders
* Flat array fixes because Elasticsearch can sometimes return `[]` or `[[]]` in-deterministically in some cases 🤷 , so we just flatten the array out completely and test for `[]` within those tests.
* `waitForSignalsToBePresent` was missing in a test and sometimes we would get an empty array response which would fail CI.

Also I audited other tests for `[[]]` and `waitForSignalsToBePresent` and fixed them where they were present or if the `waitForSignalsToBePresent` count was incorrect. This should give us more stability when the CI is under pressure.

Sorting fixes:
#115554
#115321
#115319
#114581


Flat array fixes:
#89052
#115315
#115308
#115304
#115313
#113418

Missing additional check for "waitForSignalsToBePresent" or incorrect number of signals to wait for fixes:
#115310


### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Frank Hassanabad <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Projects
None yet
Development

No branches or pull requests

4 participants