Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Security Solution Cypress.x-pack/plugins/security_solution/public/management/cypress/e2e/response_actions/response_console·cy·ts - Response console User journey for Isolate command: isolate and release an endpoint should release host from response console should release host from response console #168390

Closed
kibanamachine opened this issue Oct 9, 2023 · 3 comments · Fixed by #168438
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test OLM Sprint Team:Defend Workflows “EDR Workflows” sub-team of Security Solution

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Oct 9, 2023

A test failed on a tracked branch

AssertionError: Timed out retrying after 60000ms: expected '<td.euiTableRowCell.euiTableRowCell--middle>' to contain text 'Isolated', but the text was 'Agent statusHealthy'
    at Context.eval (webpack:///./tasks/isolate.ts:128:27)

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 9, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 9, 2023
@kibanamachine kibanamachine added the Team:Defend Workflows “EDR Workflows” sub-team of Security Solution label Oct 9, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 9, 2023
szwarckonrad added a commit that referenced this issue Oct 11, 2023
This pull request extends the agent fleet check timeout from 2 minutes
to 4 minutes. We've identified a number of unreliable tests that fail
during the `beforeAll` stage while executing the `createEndpointHost`
task. The following logs appear before the timeout:

```
info Enrolling Elastic Agent with Fleet
  | Installing service....... DONE
  | Starting service... DONE
  | Enrolling Elastic Agent with Fleet..........Successfully enrolled the Elastic Agent.
  | Elastic Agent has been successfully installed.
  | info Waiting for Agent to check-in with Fleet
```

The error message we encounter is `> Timed out waiting for host
[test-host-4981] to appear in Fleet.`

It appears that all the preceding steps are successful, and only the
final one fails due to either the agent not checking in with the fleet
for 2 minutes or the agent being unhealthy for two minutes. Since I
haven't been able to replicate this behavior locally, and there isn't a
way to inspect what's happening on the agent, I believe the best course
of action at this point is to extend the timeout and monitor the
results.

Reports of this error:
closes #168427
closes #168394
closes #168393
closes #168390
closes #168363
closes #168362
closes #168361
closes #168360
closes #168359

Affected CI runs:
https://buildkite.com/elastic/kibana-on-merge/builds/36483
https://buildkite.com/elastic/kibana-on-merge/builds/36497
https://buildkite.com/elastic/kibana-on-merge/builds/36501
https://buildkite.com/elastic/kibana-on-merge/builds/36526

Another time out happens from time to time when previously set 10
minutes timeout on `createEndpointHost` task is not enough to set up the
environment. Its portrayed below, timeout happens during agent setup
```
  | default: Running: inline script
  | default: Reading package lists...
  | default: Building dependency tree...
  | default: Reading state information...
  | default: Suggested packages:
  | default:   zip
  | default: The following NEW packages will be installed:
  | default:   unzip
  | default: 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
  | default: Need to get 174 kB of archives.
  | default: After this operation, 385 kB of additional disk space will be used.
  | default: Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB]
  | default: dpkg-preconfigure: unable to re-open stdin: No such file or directory
  | default: Fetched 174 kB in 1s (210 kB/s)
  | default: Selecting previously unselected package unzip.
  | (Reading database ... 63961 files and directories currently installed.)
  | default: Preparing to unpack .../unzip_6.0-26ubuntu3.1_amd64.deb ...
  | default: Unpacking unzip (6.0-26ubuntu3.1) ...
  | default: Setting up unzip (6.0-26ubuntu3.1) ...
  | default: Processing triggers for man-db (2.10.2-1) ...
  |  
  | CypressError: `cy.task('createEndpointHost')` timed out after waiting `600000ms`.
```
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 11, 2023
…168438)

This pull request extends the agent fleet check timeout from 2 minutes
to 4 minutes. We've identified a number of unreliable tests that fail
during the `beforeAll` stage while executing the `createEndpointHost`
task. The following logs appear before the timeout:

```
info Enrolling Elastic Agent with Fleet
  | Installing service....... DONE
  | Starting service... DONE
  | Enrolling Elastic Agent with Fleet..........Successfully enrolled the Elastic Agent.
  | Elastic Agent has been successfully installed.
  | info Waiting for Agent to check-in with Fleet
```

The error message we encounter is `> Timed out waiting for host
[test-host-4981] to appear in Fleet.`

It appears that all the preceding steps are successful, and only the
final one fails due to either the agent not checking in with the fleet
for 2 minutes or the agent being unhealthy for two minutes. Since I
haven't been able to replicate this behavior locally, and there isn't a
way to inspect what's happening on the agent, I believe the best course
of action at this point is to extend the timeout and monitor the
results.

Reports of this error:
closes elastic#168427
closes elastic#168394
closes elastic#168393
closes elastic#168390
closes elastic#168363
closes elastic#168362
closes elastic#168361
closes elastic#168360
closes elastic#168359

Affected CI runs:
https://buildkite.com/elastic/kibana-on-merge/builds/36483
https://buildkite.com/elastic/kibana-on-merge/builds/36497
https://buildkite.com/elastic/kibana-on-merge/builds/36501
https://buildkite.com/elastic/kibana-on-merge/builds/36526

Another time out happens from time to time when previously set 10
minutes timeout on `createEndpointHost` task is not enough to set up the
environment. Its portrayed below, timeout happens during agent setup
```
  | default: Running: inline script
  | default: Reading package lists...
  | default: Building dependency tree...
  | default: Reading state information...
  | default: Suggested packages:
  | default:   zip
  | default: The following NEW packages will be installed:
  | default:   unzip
  | default: 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
  | default: Need to get 174 kB of archives.
  | default: After this operation, 385 kB of additional disk space will be used.
  | default: Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB]
  | default: dpkg-preconfigure: unable to re-open stdin: No such file or directory
  | default: Fetched 174 kB in 1s (210 kB/s)
  | default: Selecting previously unselected package unzip.
  | (Reading database ... 63961 files and directories currently installed.)
  | default: Preparing to unpack .../unzip_6.0-26ubuntu3.1_amd64.deb ...
  | default: Unpacking unzip (6.0-26ubuntu3.1) ...
  | default: Setting up unzip (6.0-26ubuntu3.1) ...
  | default: Processing triggers for man-db (2.10.2-1) ...
  |  
  | CypressError: `cy.task('createEndpointHost')` timed out after waiting `600000ms`.
```

(cherry picked from commit 91cdbe2)
kibanamachine referenced this issue Oct 11, 2023
…168438) (#168614)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[EDR Workflows][E2E] Increase the timeout of agent check in
(#168438)](#168438)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Konrad
Szwarc","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-11T14:26:45Z","message":"[EDR
Workflows][E2E] Increase the timeout of agent check in (#168438)\n\nThis
pull request extends the agent fleet check timeout from 2 minutes\r\nto
4 minutes. We've identified a number of unreliable tests that
fail\r\nduring the `beforeAll` stage while executing the
`createEndpointHost`\r\ntask. The following logs appear before the
timeout:\r\n\r\n```\r\ninfo Enrolling Elastic Agent with Fleet\r\n |
Installing service....... DONE\r\n | Starting service... DONE\r\n |
Enrolling Elastic Agent with Fleet..........Successfully enrolled the
Elastic Agent.\r\n | Elastic Agent has been successfully installed.\r\n
| info Waiting for Agent to check-in with Fleet\r\n```\r\n\r\nThe error
message we encounter is `> Timed out waiting for
host\r\n[test-host-4981] to appear in Fleet.`\r\n\r\nIt appears that all
the preceding steps are successful, and only the\r\nfinal one fails due
to either the agent not checking in with the fleet\r\nfor 2 minutes or
the agent being unhealthy for two minutes. Since I\r\nhaven't been able
to replicate this behavior locally, and there isn't a\r\nway to inspect
what's happening on the agent, I believe the best course\r\nof action at
this point is to extend the timeout and monitor
the\r\nresults.\r\n\r\nReports of this error:\r\ncloses
https://github.com/elastic/kibana/issues/168427\r\ncloses
https://github.com/elastic/kibana/issues/168394\r\ncloses
https://github.com/elastic/kibana/issues/168393\r\ncloses
https://github.com/elastic/kibana/issues/168390\r\ncloses
https://github.com/elastic/kibana/issues/168363\r\ncloses
https://github.com/elastic/kibana/issues/168362\r\ncloses
https://github.com/elastic/kibana/issues/168361\r\ncloses
https://github.com/elastic/kibana/issues/168360\r\ncloses
https://github.com/elastic/kibana/issues/168359\r\n\r\nAffected CI
runs:\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36483\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36497\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36501\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36526\r\n\r\nAnother
time out happens from time to time when previously set 10\r\nminutes
timeout on `createEndpointHost` task is not enough to set up
the\r\nenvironment. Its portrayed below, timeout happens during agent
setup\r\n```\r\n  | default: Running: inline script\r\n  | default:
Reading package lists...\r\n  | default: Building dependency
tree...\r\n  | default: Reading state information...\r\n  | default:
Suggested packages:\r\n  | default: zip\r\n  | default: The following
NEW packages will be installed:\r\n  | default: unzip\r\n  | default: 0
upgraded, 1 newly installed, 0 to remove and 0 not upgraded.\r\n  |
default: Need to get 174 kB of archives.\r\n  | default: After this
operation, 385 kB of additional disk space will be used.\r\n  | default:
Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip
amd64 6.0-26ubuntu3.1 [174 kB]\r\n  | default: dpkg-preconfigure: unable
to re-open stdin: No such file or directory\r\n  | default: Fetched 174
kB in 1s (210 kB/s)\r\n  | default: Selecting previously unselected
package unzip.\r\n  | (Reading database ... 63961 files and directories
currently installed.)\r\n  | default: Preparing to unpack
.../unzip_6.0-26ubuntu3.1_amd64.deb ...\r\n  | default: Unpacking unzip
(6.0-26ubuntu3.1) ...\r\n  | default: Setting up unzip (6.0-26ubuntu3.1)
...\r\n  | default: Processing triggers for man-db (2.10.2-1) ...\r\n  |
 \r\n  | CypressError: `cy.task('createEndpointHost')` timed out after
waiting
`600000ms`.\r\n```","sha":"91cdbe2d354100683b5d8670de88e0b2cf665ba9","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","v8.11.0","v8.12.0"],"number":168438,"url":"https://github.com/elastic/kibana/pull/168438","mergeCommit":{"message":"[EDR
Workflows][E2E] Increase the timeout of agent check in (#168438)\n\nThis
pull request extends the agent fleet check timeout from 2 minutes\r\nto
4 minutes. We've identified a number of unreliable tests that
fail\r\nduring the `beforeAll` stage while executing the
`createEndpointHost`\r\ntask. The following logs appear before the
timeout:\r\n\r\n```\r\ninfo Enrolling Elastic Agent with Fleet\r\n |
Installing service....... DONE\r\n | Starting service... DONE\r\n |
Enrolling Elastic Agent with Fleet..........Successfully enrolled the
Elastic Agent.\r\n | Elastic Agent has been successfully installed.\r\n
| info Waiting for Agent to check-in with Fleet\r\n```\r\n\r\nThe error
message we encounter is `> Timed out waiting for
host\r\n[test-host-4981] to appear in Fleet.`\r\n\r\nIt appears that all
the preceding steps are successful, and only the\r\nfinal one fails due
to either the agent not checking in with the fleet\r\nfor 2 minutes or
the agent being unhealthy for two minutes. Since I\r\nhaven't been able
to replicate this behavior locally, and there isn't a\r\nway to inspect
what's happening on the agent, I believe the best course\r\nof action at
this point is to extend the timeout and monitor
the\r\nresults.\r\n\r\nReports of this error:\r\ncloses
https://github.com/elastic/kibana/issues/168427\r\ncloses
https://github.com/elastic/kibana/issues/168394\r\ncloses
https://github.com/elastic/kibana/issues/168393\r\ncloses
https://github.com/elastic/kibana/issues/168390\r\ncloses
https://github.com/elastic/kibana/issues/168363\r\ncloses
https://github.com/elastic/kibana/issues/168362\r\ncloses
https://github.com/elastic/kibana/issues/168361\r\ncloses
https://github.com/elastic/kibana/issues/168360\r\ncloses
https://github.com/elastic/kibana/issues/168359\r\n\r\nAffected CI
runs:\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36483\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36497\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36501\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36526\r\n\r\nAnother
time out happens from time to time when previously set 10\r\nminutes
timeout on `createEndpointHost` task is not enough to set up
the\r\nenvironment. Its portrayed below, timeout happens during agent
setup\r\n```\r\n  | default: Running: inline script\r\n  | default:
Reading package lists...\r\n  | default: Building dependency
tree...\r\n  | default: Reading state information...\r\n  | default:
Suggested packages:\r\n  | default: zip\r\n  | default: The following
NEW packages will be installed:\r\n  | default: unzip\r\n  | default: 0
upgraded, 1 newly installed, 0 to remove and 0 not upgraded.\r\n  |
default: Need to get 174 kB of archives.\r\n  | default: After this
operation, 385 kB of additional disk space will be used.\r\n  | default:
Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip
amd64 6.0-26ubuntu3.1 [174 kB]\r\n  | default: dpkg-preconfigure: unable
to re-open stdin: No such file or directory\r\n  | default: Fetched 174
kB in 1s (210 kB/s)\r\n  | default: Selecting previously unselected
package unzip.\r\n  | (Reading database ... 63961 files and directories
currently installed.)\r\n  | default: Preparing to unpack
.../unzip_6.0-26ubuntu3.1_amd64.deb ...\r\n  | default: Unpacking unzip
(6.0-26ubuntu3.1) ...\r\n  | default: Setting up unzip (6.0-26ubuntu3.1)
...\r\n  | default: Processing triggers for man-db (2.10.2-1) ...\r\n  |
 \r\n  | CypressError: `cy.task('createEndpointHost')` timed out after
waiting
`600000ms`.\r\n```","sha":"91cdbe2d354100683b5d8670de88e0b2cf665ba9"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/168438","number":168438,"mergeCommit":{"message":"[EDR
Workflows][E2E] Increase the timeout of agent check in (#168438)\n\nThis
pull request extends the agent fleet check timeout from 2 minutes\r\nto
4 minutes. We've identified a number of unreliable tests that
fail\r\nduring the `beforeAll` stage while executing the
`createEndpointHost`\r\ntask. The following logs appear before the
timeout:\r\n\r\n```\r\ninfo Enrolling Elastic Agent with Fleet\r\n |
Installing service....... DONE\r\n | Starting service... DONE\r\n |
Enrolling Elastic Agent with Fleet..........Successfully enrolled the
Elastic Agent.\r\n | Elastic Agent has been successfully installed.\r\n
| info Waiting for Agent to check-in with Fleet\r\n```\r\n\r\nThe error
message we encounter is `> Timed out waiting for
host\r\n[test-host-4981] to appear in Fleet.`\r\n\r\nIt appears that all
the preceding steps are successful, and only the\r\nfinal one fails due
to either the agent not checking in with the fleet\r\nfor 2 minutes or
the agent being unhealthy for two minutes. Since I\r\nhaven't been able
to replicate this behavior locally, and there isn't a\r\nway to inspect
what's happening on the agent, I believe the best course\r\nof action at
this point is to extend the timeout and monitor
the\r\nresults.\r\n\r\nReports of this error:\r\ncloses
https://github.com/elastic/kibana/issues/168427\r\ncloses
https://github.com/elastic/kibana/issues/168394\r\ncloses
https://github.com/elastic/kibana/issues/168393\r\ncloses
https://github.com/elastic/kibana/issues/168390\r\ncloses
https://github.com/elastic/kibana/issues/168363\r\ncloses
https://github.com/elastic/kibana/issues/168362\r\ncloses
https://github.com/elastic/kibana/issues/168361\r\ncloses
https://github.com/elastic/kibana/issues/168360\r\ncloses
https://github.com/elastic/kibana/issues/168359\r\n\r\nAffected CI
runs:\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36483\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36497\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36501\r\nhttps://buildkite.com/elastic/kibana-on-merge/builds/36526\r\n\r\nAnother
time out happens from time to time when previously set 10\r\nminutes
timeout on `createEndpointHost` task is not enough to set up
the\r\nenvironment. Its portrayed below, timeout happens during agent
setup\r\n```\r\n  | default: Running: inline script\r\n  | default:
Reading package lists...\r\n  | default: Building dependency
tree...\r\n  | default: Reading state information...\r\n  | default:
Suggested packages:\r\n  | default: zip\r\n  | default: The following
NEW packages will be installed:\r\n  | default: unzip\r\n  | default: 0
upgraded, 1 newly installed, 0 to remove and 0 not upgraded.\r\n  |
default: Need to get 174 kB of archives.\r\n  | default: After this
operation, 385 kB of additional disk space will be used.\r\n  | default:
Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip
amd64 6.0-26ubuntu3.1 [174 kB]\r\n  | default: dpkg-preconfigure: unable
to re-open stdin: No such file or directory\r\n  | default: Fetched 174
kB in 1s (210 kB/s)\r\n  | default: Selecting previously unselected
package unzip.\r\n  | (Reading database ... 63961 files and directories
currently installed.)\r\n  | default: Preparing to unpack
.../unzip_6.0-26ubuntu3.1_amd64.deb ...\r\n  | default: Unpacking unzip
(6.0-26ubuntu3.1) ...\r\n  | default: Setting up unzip (6.0-26ubuntu3.1)
...\r\n  | default: Processing triggers for man-db (2.10.2-1) ...\r\n  |
 \r\n  | CypressError: `cy.task('createEndpointHost')` timed out after
waiting
`600000ms`.\r\n```","sha":"91cdbe2d354100683b5d8670de88e0b2cf665ba9"}}]}]
BACKPORT-->

Co-authored-by: Konrad Szwarc <[email protected]>
@kibanamachine kibanamachine reopened this Oct 12, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

dej611 pushed a commit to dej611/kibana that referenced this issue Oct 17, 2023
…168438)

This pull request extends the agent fleet check timeout from 2 minutes
to 4 minutes. We've identified a number of unreliable tests that fail
during the `beforeAll` stage while executing the `createEndpointHost`
task. The following logs appear before the timeout:

```
info Enrolling Elastic Agent with Fleet
  | Installing service....... DONE
  | Starting service... DONE
  | Enrolling Elastic Agent with Fleet..........Successfully enrolled the Elastic Agent.
  | Elastic Agent has been successfully installed.
  | info Waiting for Agent to check-in with Fleet
```

The error message we encounter is `> Timed out waiting for host
[test-host-4981] to appear in Fleet.`

It appears that all the preceding steps are successful, and only the
final one fails due to either the agent not checking in with the fleet
for 2 minutes or the agent being unhealthy for two minutes. Since I
haven't been able to replicate this behavior locally, and there isn't a
way to inspect what's happening on the agent, I believe the best course
of action at this point is to extend the timeout and monitor the
results.

Reports of this error:
closes elastic#168427
closes elastic#168394
closes elastic#168393
closes elastic#168390
closes elastic#168363
closes elastic#168362
closes elastic#168361
closes elastic#168360
closes elastic#168359

Affected CI runs:
https://buildkite.com/elastic/kibana-on-merge/builds/36483
https://buildkite.com/elastic/kibana-on-merge/builds/36497
https://buildkite.com/elastic/kibana-on-merge/builds/36501
https://buildkite.com/elastic/kibana-on-merge/builds/36526

Another time out happens from time to time when previously set 10
minutes timeout on `createEndpointHost` task is not enough to set up the
environment. Its portrayed below, timeout happens during agent setup
```
  | default: Running: inline script
  | default: Reading package lists...
  | default: Building dependency tree...
  | default: Reading state information...
  | default: Suggested packages:
  | default:   zip
  | default: The following NEW packages will be installed:
  | default:   unzip
  | default: 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
  | default: Need to get 174 kB of archives.
  | default: After this operation, 385 kB of additional disk space will be used.
  | default: Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB]
  | default: dpkg-preconfigure: unable to re-open stdin: No such file or directory
  | default: Fetched 174 kB in 1s (210 kB/s)
  | default: Selecting previously unselected package unzip.
  | (Reading database ... 63961 files and directories currently installed.)
  | default: Preparing to unpack .../unzip_6.0-26ubuntu3.1_amd64.deb ...
  | default: Unpacking unzip (6.0-26ubuntu3.1) ...
  | default: Setting up unzip (6.0-26ubuntu3.1) ...
  | default: Processing triggers for man-db (2.10.2-1) ...
  |  
  | CypressError: `cy.task('createEndpointHost')` timed out after waiting `600000ms`.
```
ashokaditya added a commit that referenced this issue Nov 1, 2023
…response console - isolate) (#169539)

## Summary

Split test to avoid long running times.

Fixes
#168296
#168390

**Flaky test runner**
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754
x 150 (3 failures for document signing: 7, 101, 141)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809
x 50 (all pass for document_signing.cy.ts)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835
x 50 (2 fails for document signing and 1 fail for response console
release due to timeout on response delay)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842
x 25 (all pass for document_singing.cy.ts and
response_console_actions.cy.ts)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843
x 25 (3 fails for document_signing.cy.ts related to timeout for action
response)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844
x 100 (all pass for release, 3 failures for document signing due to time
out waiting for action response)

-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848
x 50 ( all pass, 1 unrelated failure related to
`endpoints_list_response_console.cy·ts`)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849
x 50 (all pass)
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
ashokaditya added a commit to ashokaditya/kibana that referenced this issue Nov 2, 2023
…response console - isolate) (elastic#169539)

## Summary

Split test to avoid long running times.

Fixes
elastic#168296
elastic#168390

**Flaky test runner**
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754
x 150 (3 failures for document signing: 7, 101, 141)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809
x 50 (all pass for document_signing.cy.ts)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835
x 50 (2 fails for document signing and 1 fail for response console
release due to timeout on response delay)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842
x 25 (all pass for document_singing.cy.ts and
response_console_actions.cy.ts)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843
x 25 (3 fails for document_signing.cy.ts related to timeout for action
response)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844
x 100 (all pass for release, 3 failures for document signing due to time
out waiting for action response)

-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848
x 50 ( all pass, 1 unrelated failure related to
`endpoints_list_response_console.cy·ts`)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849
x 50 (all pass)
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

(cherry picked from commit a3d71f2)

# Conflicts:
#	x-pack/plugins/security_solution/public/management/cypress/e2e/response_actions/response_console_actions.cy.ts
ashokaditya added a commit that referenced this issue Nov 2, 2023
…ning & response console - isolate) (#169539) (#170381)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution][Endpoint] Unskip flaky tests (document signing &
response console - isolate)
(#169539)](#169539)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT
[{"author":{"name":"Ash","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-11-01T20:33:36Z","message":"[Security
Solution][Endpoint] Unskip flaky tests (document signing & response
console - isolate) (#169539)\n\n## Summary\r\n\r\nSplit test to avoid
long running
times.\r\n\r\nFixes\r\nhttps://github.com//issues/168296\r\nhttps://github.com//issues/168390\r\n\r\n**Flaky
test
runner**\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754\r\nx
150 (3 failures for document signing: 7, 101,
141)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809\r\nx
50 (all pass for
document_signing.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835\r\nx
50 (2 fails for document signing and 1 fail for response
console\r\nrelease due to timeout on response
delay)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842\r\nx
25 (all pass for document_singing.cy.ts
and\r\nresponse_console_actions.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843\r\nx
25 (3 fails for document_signing.cy.ts related to timeout for
action\r\nresponse)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844\r\nx
100 (all pass for release, 3 failures for document signing due to
time\r\nout waiting for action
response)\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848\r\nx
50 ( all pass, 1 unrelated failure related
to\r\n`endpoints_list_response_console.cy·ts`)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849\r\nx
50 (all pass)\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"a3d71f2534a56dd070982d6d8c60e56aa43ae2fa","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","OLM
Sprint","v8.11.0","v8.12.0"],"number":169539,"url":"https://github.com/elastic/kibana/pull/169539","mergeCommit":{"message":"[Security
Solution][Endpoint] Unskip flaky tests (document signing & response
console - isolate) (#169539)\n\n## Summary\r\n\r\nSplit test to avoid
long running
times.\r\n\r\nFixes\r\nhttps://github.com//issues/168296\r\nhttps://github.com//issues/168390\r\n\r\n**Flaky
test
runner**\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754\r\nx
150 (3 failures for document signing: 7, 101,
141)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809\r\nx
50 (all pass for
document_signing.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835\r\nx
50 (2 fails for document signing and 1 fail for response
console\r\nrelease due to timeout on response
delay)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842\r\nx
25 (all pass for document_singing.cy.ts
and\r\nresponse_console_actions.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843\r\nx
25 (3 fails for document_signing.cy.ts related to timeout for
action\r\nresponse)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844\r\nx
100 (all pass for release, 3 failures for document signing due to
time\r\nout waiting for action
response)\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848\r\nx
50 ( all pass, 1 unrelated failure related
to\r\n`endpoints_list_response_console.cy·ts`)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849\r\nx
50 (all pass)\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"a3d71f2534a56dd070982d6d8c60e56aa43ae2fa"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169539","number":169539,"mergeCommit":{"message":"[Security
Solution][Endpoint] Unskip flaky tests (document signing & response
console - isolate) (#169539)\n\n## Summary\r\n\r\nSplit test to avoid
long running
times.\r\n\r\nFixes\r\nhttps://github.com//issues/168296\r\nhttps://github.com//issues/168390\r\n\r\n**Flaky
test
runner**\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754\r\nx
150 (3 failures for document signing: 7, 101,
141)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809\r\nx
50 (all pass for
document_signing.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835\r\nx
50 (2 fails for document signing and 1 fail for response
console\r\nrelease due to timeout on response
delay)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842\r\nx
25 (all pass for document_singing.cy.ts
and\r\nresponse_console_actions.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843\r\nx
25 (3 fails for document_signing.cy.ts related to timeout for
action\r\nresponse)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844\r\nx
100 (all pass for release, 3 failures for document signing due to
time\r\nout waiting for action
response)\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848\r\nx
50 ( all pass, 1 unrelated failure related
to\r\n`endpoints_list_response_console.cy·ts`)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849\r\nx
50 (all pass)\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"a3d71f2534a56dd070982d6d8c60e56aa43ae2fa"}}]}]
BACKPORT-->
@ashokaditya
Copy link
Member

PR #169539

delanni pushed a commit to delanni/kibana that referenced this issue Nov 6, 2023
…response console - isolate) (elastic#169539)

## Summary

Split test to avoid long running times.

Fixes
elastic#168296
elastic#168390

**Flaky test runner**
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754
x 150 (3 failures for document signing: 7, 101, 141)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809
x 50 (all pass for document_signing.cy.ts)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835
x 50 (2 fails for document signing and 1 fail for response console
release due to timeout on response delay)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842
x 25 (all pass for document_singing.cy.ts and
response_console_actions.cy.ts)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843
x 25 (3 fails for document_signing.cy.ts related to timeout for action
response)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844
x 100 (all pass for release, 3 failures for document signing due to time
out waiting for action response)

-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848
x 50 ( all pass, 1 unrelated failure related to
`endpoints_list_response_console.cy·ts`)
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849
x 50 (all pass)
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test OLM Sprint Team:Defend Workflows “EDR Workflows” sub-team of Security Solution
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants