Upgrades cypress to 13.15.1 #198715

michaelolo24 · 2024-11-01T18:22:32Z

Summary

This PR upgrades cypress to 13.15.1. This is needed to fix an very annoying issue where Cypress tests hangs when ran locally and when the window is not focused

See Cypress issue here: cypress-io/cypress#28392

elasticmachine · 2024-11-01T18:22:35Z

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine · 2024-11-05T21:12:02Z

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

github-actions · 2024-11-05T21:12:11Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

cauemarcondes · 2024-11-07T14:55:41Z

@michaelolo24 The APM cypress tests failing is a known issue and we'll fix that soon. Sorry for that.

michaelolo24 · 2024-11-07T14:57:07Z

@michaelolo24 The APM cypress tests failing is a known issue and we'll fix that soon. Sorry for that.

Thanks @cauemarcondes ! Since I would rather not skip tests, I'll wait for those issues to be fixed and then will push this upgrade. Thank you!

michaelolo24 · 2024-11-07T16:28:34Z

@cauemarcondes - We actually may end up skipping them to get the upgrade in as running cypress locally has become increasingly difficult. I tried to be judicious in skipping only the specific tests that we're failing rather than whole blocks.

If it helps, I think the fix for service_overview, service_inventory, and agent_configuration files may involve either updating the input/combobox components for environmentFilter and serviceNameComboBox to be more easily modifiable or updating the tests to not rely on typing in them

For the integration_polciy and tutorial files, the pages don't seem to be loading the views properly so not sure what's happening there

cauemarcondes · 2024-11-07T16:53:29Z

I've got a PR skipping the problematic tests: #199336

cauemarcondes · 2024-11-08T10:54:36Z

@michaelolo24 my PR has been merged.

PhilippeOberti · 2024-11-12T16:39:51Z

@michaelolo24 my PR has been merged.

thanks! I rebased this PR accordingly

PhilippeOberti · 2024-11-12T22:24:12Z

@cauemarcondes can you take one more look and review this PR? The build is green it's ready to go and we need an approval from the @elastic/obs-ux-infra_services-team.
Thanks!

cauemarcondes · 2024-11-13T10:37:15Z

...ins/observability_solution/apm/ftr_e2e/cypress/e2e/service_inventory/service_inventory.cy.ts

@@ -88,7 +88,7 @@ describe('Service inventory', () => {
      cy.visitKibana(serviceInventoryHref);
    });

-    it('with the correct environment when changing the environment', () => {
+    it.skip('with the correct environment when changing the environment', () => {


@michaelolo24 these tests are not failing on main AFAICT

I unskipped all the tests, let's see what the build says!

@cauemarcondes over the last 2 builds (this one and that one) we have 3 tests failing:

service_inventory/service_inventory.cy.ts

service_overview/service_overview.cy.ts

settings/agent_configurations.cy.ts

These are the 3 tests that @michaelolo24 had skipped and I unskipped in my last commit.
If these are not failing on main, can you help us figuring out why these 3 tests would consistently fail?

@PhilippeOberti / @michaelolo24 after checking out your branch and running the test I could not see any specific problem with the tests you mentioned. They are breaking, as CI pointed, but there is no clear reason why. This is the error showed on the console:

2024-11-18 11:58:49.047 Cypress Helper (Renderer)[30120:5226514] CoreText note: Set a breakpoint on CTFontLogSystemFontNameRequest to debug. We detected that the Electron Renderer process just crashed. We have failed the current spec but will continue running the next spec. This can happen for a number of different reasons. If you're running lots of tests on a memory intense application. - Try increasing the CPU/memory on the machine you're running on. - Try enabling experimentalMemoryManagement in your config file. - Try lowering numTestsKeptInMemory in your config file during 'cypress open'. You can learn more here: https://on.cypress.io/renderer-process-crashed (Results) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Tests: 1 │ │ Passing: 0 │ │ Failing: 1 │ │ Pending: 0 │ │ Skipped: 0 │ │ Screenshots: 0 │ │ Video: true │ │ Duration: 11 seconds │ │ Spec Ran: service_inventory.cy.ts │ └────────────────────────────────────────────────────────────────────────────────────────────────┘

I double-cheked and the tests are indeed passing on the main branch.

I remember @michaelolo24 added this experimentalMemoryManagement to the config file (here) but I think Glo said it wasn't in the right place.
I just pushed a commit moving it to the e2e location, hopefully that will do the trick here... if not I don't know what to do...

considering there are only 3 files modified in this PR, 2 of them are package.json and its lock file, and the third one being the experimentalMemoryManagement change to the config file to try to fix the failing tests, I think it's safe to say that the failures are due to the Cypress update...
If we want to continue updating Cypress, I think we need to re-work these 3 tests a bit to make them work with the newer version.

This version works. Although we're not doing like the previous test, we are just opening the combo box and selecting the element.

it('with the correct environment when changing the environment', () => { cy.intercept('GET', '/internal/apm/suggestions?*').as('environmentSuggestionOptions'); cy.wait(mainAliasNames); cy.wait('@environmentSuggestionOptions'); cy.getByTestSubj('environmentFilter').click(); cy.contains('button', 'production').click(); cy.expectAPIsToHaveBeenCalledWith({ apisIntercepted: mainAliasNames, value: 'environment=production', }); });

Thanks @cauemarcondes, makes sense ! Would you be willing to commit the changes for the three files directly to the PR?

Hey @michaelolo24 I was trying to fix the tests but the browser keeps crasing all the time with this message:

We detected that the Chrome Renderer process just crashed. We have failed the current spec but will continue running the next spec. This can happen for a number of different reasons. If you're running lots of tests on a memory intense application. - Try increasing the CPU/memory on the machine you're running on. - Try enabling experimentalMemoryManagement in your config file. - Try lowering numTestsKeptInMemory in your config file during 'cypress open'. You can learn more here: https://on.cypress.io/renderer-process-crashed

Unfortunatly I don't feel comfortable approving this PR until this is fixed.

Yea, this goes back to the error I was seeing when interacting with the environmentFilter dropdown. Suspecting cypress crashes due to an infinite loop of trying to delete the field values, but not being able to delete...but not 100% sure. I was able to get the changes I made working locally, but because of the way the component works, it's hard to properly test the suggestion api since you can't clear the environment field. We'll see if this current ci run passes. If so, please feel free to make modifications you'd like to make to my changes to better fit APM's expectations as the tests shouldn't crash any longer 🤞🏾

kibanamachine · 2024-11-25T21:34:32Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7478

[❌] APM - Cypress: 20/25 tests passed.

see run history

elasticmachine · 2024-11-26T02:00:50Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 117565d
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-198715-117565d9c729

Failed CI Steps

Jest Tests #19

Test Failures

[job] [logs] Jest Tests #19 / Category can submit without setting a category

Metrics [docs]

✅ unchanged

History

💚 Build #254225 succeeded 16972f2
💚 Build #253653 succeeded 68d38eb
💔 Build #253623 failed 99ac2ff
💛 Build #253366 was flaky c87d094
💔 Build #251976 failed b412cf9

kibanamachine · 2024-11-26T02:32:37Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7480

[✅] APM - Cypress: 25/25 tests passed.

see run history

cauemarcondes

LGTM

## Summary This PR upgrades cypress to `13.15.1`. This is needed to fix an very annoying issue where Cypress tests hangs when ran locally and when the window is not focused See Cypress issue here: cypress-io/cypress#28392 --------- Co-authored-by: PhilippeOberti <[email protected]>

michaelolo24 added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. ci:all-cypress-suites v8.17.0 labels Nov 1, 2024

michaelolo24 requested a review from MadameSheema November 1, 2024 18:22

michaelolo24 mentioned this pull request Nov 1, 2024

[Security Solution][Threat Hunting] - Unskip threat hunting tests #195662

Merged

PhilippeOberti approved these changes Nov 1, 2024

View reviewed changes

michaelolo24 force-pushed the upgrade-cypress branch 2 times, most recently from ae5760c to 574336a Compare November 5, 2024 21:11

michaelolo24 requested a review from a team as a code owner November 5, 2024 21:11

botelastic bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Nov 5, 2024

michaelolo24 force-pushed the upgrade-cypress branch 3 times, most recently from 2177bcc to 4c23685 Compare November 7, 2024 00:11

michaelolo24 force-pushed the upgrade-cypress branch from 3f13009 to 3261fd1 Compare November 7, 2024 16:20

michaelolo24 force-pushed the upgrade-cypress branch from 3261fd1 to a7c48ac Compare November 7, 2024 16:37

PhilippeOberti force-pushed the upgrade-cypress branch from a7c48ac to 73fbb46 Compare November 12, 2024 16:39

cauemarcondes requested changes Nov 13, 2024

View reviewed changes

michaelolo24 and others added 11 commits November 25, 2024 19:52

Upgrade cypress

4d26f47

bump cypress patch

3e95d8b

skip known failing apm tests

bc4272a

fix conflict

b95ade6

unskiping apm tests

e5b241b

trying to move experimentalMemoryManagement in the correct location

e488721

address apm tests

c3a135a

fix last cypress test

68c2f77

remove .only

c79022b

ensure options list visibility

ed933a1

chain assertion

117565d

michaelolo24 force-pushed the upgrade-cypress branch from 16972f2 to 117565d Compare November 26, 2024 00:52

michaelolo24 added v9.0.0 v8.18.0 and removed v8.17.0 labels Nov 26, 2024

cauemarcondes approved these changes Nov 26, 2024

View reviewed changes

michaelolo24 merged commit 1865869 into elastic:main Nov 26, 2024
63 checks passed

kibanamachine mentioned this pull request Dec 11, 2024

Upgrade mocha to 10.3.0 #203500

Merged

This was referenced Dec 13, 2024

Update dependency @elastic/elasticsearch to ^8.16.0 (main) #200275

Merged

Sustainable Kibana Architecture: Move modules owned by @elastic/obs-knowledge-team #202766

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrades cypress to 13.15.1 #198715

Upgrades cypress to 13.15.1 #198715

michaelolo24 commented Nov 1, 2024 •

edited by PhilippeOberti

Loading

elasticmachine commented Nov 1, 2024

elasticmachine commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

cauemarcondes commented Nov 7, 2024

michaelolo24 commented Nov 7, 2024

michaelolo24 commented Nov 7, 2024

cauemarcondes commented Nov 7, 2024

cauemarcondes commented Nov 8, 2024

PhilippeOberti commented Nov 12, 2024

PhilippeOberti commented Nov 12, 2024

cauemarcondes Nov 13, 2024

PhilippeOberti Nov 13, 2024

PhilippeOberti Nov 14, 2024 •

edited

Loading

cauemarcondes Nov 18, 2024 •

edited

Loading

PhilippeOberti Nov 18, 2024

PhilippeOberti Nov 18, 2024

cauemarcondes Nov 18, 2024

michaelolo24 Nov 18, 2024

cauemarcondes Nov 19, 2024

michaelolo24 Nov 21, 2024

kibanamachine commented Nov 25, 2024

elasticmachine commented Nov 26, 2024 •

edited

Loading

kibanamachine commented Nov 26, 2024

cauemarcondes left a comment

Upgrades cypress to 13.15.1 #198715

Upgrades cypress to 13.15.1 #198715

Conversation

michaelolo24 commented Nov 1, 2024 • edited by PhilippeOberti Loading

Summary

elasticmachine commented Nov 1, 2024

elasticmachine commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

🤖 GitHub comments

cauemarcondes commented Nov 7, 2024

michaelolo24 commented Nov 7, 2024

michaelolo24 commented Nov 7, 2024

cauemarcondes commented Nov 7, 2024

cauemarcondes commented Nov 8, 2024

PhilippeOberti commented Nov 12, 2024

PhilippeOberti commented Nov 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PhilippeOberti Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

cauemarcondes Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibanamachine commented Nov 25, 2024

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7478

elasticmachine commented Nov 26, 2024 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

kibanamachine commented Nov 26, 2024

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7480

cauemarcondes left a comment

Choose a reason for hiding this comment

michaelolo24 commented Nov 1, 2024 •

edited by PhilippeOberti

Loading

PhilippeOberti Nov 14, 2024 •

edited

Loading

cauemarcondes Nov 18, 2024 •

edited

Loading

elasticmachine commented Nov 26, 2024 •

edited

Loading