Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Canvas] Fix expression editor test #146466

Merged
merged 3 commits into from
Nov 30, 2022

Conversation

nickpeihl
Copy link
Member

@nickpeihl nickpeihl commented Nov 28, 2022

Fixes a skipped flaky test for the expression editor.

The test was skipped before the expression editor was changed to use Monaco in #133318. I refactored the test to use the Monaco service which should also address #133318 (comment).

Flaky test runner

@nickpeihl nickpeihl changed the title [Canvas] Enable flaky test [Canvas] Fix expression editor test Nov 29, 2022
@nickpeihl nickpeihl added Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas release_note:skip Skip the PR/issue when compiling release notes Feature:Canvas v8.7.0 labels Nov 29, 2022
@nickpeihl nickpeihl marked this pull request as ready for review November 29, 2022 18:10
@nickpeihl nickpeihl requested a review from a team as a code owner November 29, 2022 18:10
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-presentation (Team:Presentation)

@nickpeihl nickpeihl added impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:medium Medium Level of Effort labels Nov 29, 2022
Copy link
Contributor

@ThomThomson ThomThomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome to see this test reinstated! Approving this based on the flaky test runner result, but ideally, more than 10 runs is better for flaky test runner, because some flakiness can be pernicious and only fail once every 100ish runs.

Left one nit.

await mdBox.type(newMd);
await find.clickByCssSelector('.canvasArg--controls .euiButton');
await monacoEditor.setCodeEditorValue(newMd, 0);
await PageObjects.common.sleep(300);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could introduce some flakiness here, how did you come to the 300 ms number here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I set 300ms as a sorta arbitrary number. If I'm understanding correctly, the line below sets a 250ms debounce on updates to the sidebar editor before calling the onChange callback. That might be enough time for the expression editor to receive the updates, or I could set this to a larger sleep.

template: templateFromReactComponent(withDebounceArg(EditorArg, 250)),

I'm running a new flaky test job with 100 runs here: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1607. Do you think that is sufficient or maybe try a larger number?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 100 runs is definitely sufficient, thank you for doing that! If they all pass, then I'd say that 300ms is long enough to sleep, especially now that I know where the 300ms number comes from.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of sleep, how about putting assert statement below in a retry? That way the functional test is not couple to a debounce value

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of sleep, how about putting assert statement below in a retry? That way the functional test is not couple to a debounce value

Ooh, good idea!

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
canvas 1.0MB 1.0MB +82.0B
Unknown metric groups

ESLint disabled in files

id before after diff
osquery 1 2 +1

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
fleet 59 65 +6
osquery 109 115 +6
securitySolution 442 448 +6
total +20

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
fleet 68 74 +6
osquery 110 117 +7
securitySolution 519 525 +6
total +21

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Canvas impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:medium Medium Level of Effort release_note:skip Skip the PR/issue when compiling release notes Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas v8.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants