[QA][fix failing test] Refactor to ui settings for timepicker

[wayneseymour]

[skip-ci]

Use kibanaServer.uiSettings.replace()
instead of PageObjects.timePicker.setDefaultAbsoluteRange()

Resolves #60559

Helps with: #113998

[kibanamachine]

💚 Build Succeeded

• Buildkite Build
• Commit: fd65ef0
• Storybooks Preview

History

• 💔 Build #135 failed 74fd8bb2280f34cf7f15a3bb7ced72353d696b66
• 💔 Build #131 failed a20a71f

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[dmlemeshko]

I like the general idea of replacing UI interaction via Webdriver with kbnServer, but I think doing it for flaky (potentially buggy) tests is not the right approach.
Particular test looks like a potential UI bug that just hard to reproduce and if we make a switch, we will never find it out.

I think we should migrate stable tests to kbnServer.uiSettings.replace usage, for flaky ones I would talk to developers and do proper investigation.

[wayneseymour]

I like the general idea of replacing UI interaction via Webdriver with kbnServer, but I think doing it for flaky (potentially buggy) tests is not the right approach. Particular test looks like a potential UI bug that just hard to reproduce and if we make a switch, we will never find it out.

I think we should migrate stable tests to kbnServer.uiSettings.replace usage, for flaky ones I would talk to developers and do proper investigation.

Add, that makes a lot of sense man. I'll follow up

[wayneseymour]

Sadly there is no codeowner for this file, in .github/CODEOWNERS.
I'll check kibana-stats

[wayneseymour]

@kobelb Hello there, my team and I are in an effort to fixup tests that are flaky due to the timepicker.

Dima brought up a good point above, and I'd like your input.

[kobelb]

+1 to what @dmlemeshko's recommendation.

In general, I think that we should be doing test setup/teardown as much as possible using kbnServer.uiSettings.replace because it's faster and can help us when writing tests in such a way that they only fail for one reason. However, there's a risk in switching to using kbnServer.uiSettings.replace when there's flakiness as there might be a difficult to reproduce bug that we're overlooking.

[wayneseymour]

+1 to what @dmlemeshko's recommendation.

In general, I think that we should be doing test setup/teardown as much as possible using kbnServer.uiSettings.replace because it's faster and can help us when writing tests in such a way that they only fail for one reason. However, there's a risk in switching to using kbnServer.uiSettings.replace when there's flakiness as there might be a difficult to reproduce bug that we're overlooking.

Ok, makes sense.
I presumed the test was the problem.
Do you think it's the app instead?
If so, whom might I contact to resolve this?

[kobelb]

Based on the CI logs from when this test failed (the last couple lines repeat for a while until it times out):

[00:20:06]                 │ debg Validating 'startDate' - expected: 'Sep 19, 2015 @ 06:31:44.000, actual: Sep 19, 2015 @ 06:31:44.000'
[00:20:06]                 │ debg Waiting up to 20000ms for Timepicker popover to close...
[00:20:06]                 │ debg TestSubjects.exists(superDatePickerAbsoluteDateInput)
[00:20:06]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="superDatePickerAbsoluteDateInput"]') with timeout=2500
[00:20:06]                 │ debg TestSubjects.exists(superDatePickerAbsoluteDateInput)
[00:20:06]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="superDatePickerAbsoluteDateInput"]') with timeout=2500
[00:20:07]                 │ debg TestSubjects.exists(superDatePickerAbsoluteDateInput)
[00:20:07]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="superDatePickerAbsoluteDateInput"]') with timeout=2500
[00:20:07]                 │ debg TestSubjects.exists(superDatePickerAbsoluteDateInput)
[00:20:07]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="superDatePickerAbsoluteDateInput"]') with timeout=2500

It looks like the timepicker was stuck in an open state. So, it's possible that this is a bug with the timepicker itself or the Discover application. I'd start with the @elastic/kibana-data-discovery team to rule out it being a bug with the Discover application before looking at the timepicker as the culprit.