Enabling Uptime and Dashboard a11y test #91017

myasonik · 2021-02-10T20:04:30Z

Dashboard tests were migrated to using the panel service which performs a lot more "is it safe to click this/wait to click this until it's ready" checks so it should hopefully be more stable now
- Also skipped a few rules that were being caused by upstream problems that we aren't responsible for
Uptime test was reenabled with no changes (recent EUI update should have resolved the issue)
- Did split one test into two just to make reporting easier to grep

Fixes #90555
Fixes #91592
Fixes #77931
Fixes #77422

myasonik · 2021-02-18T15:25:01Z

Flaky test runner results aren't great 😕

In 42 runs:

4 Uptime failures at "alert popover controls"
14 Dashboard failures at "can clone panel "

myasonik · 2021-02-18T19:28:10Z

Flaky test update:

0 uptime failures 🎉
22 dashboard failures 😢

myasonik · 2021-02-18T21:22:25Z

Flaky test update: 14 dashboard failures 😕

myasonik · 2021-02-19T01:15:36Z

Flaky test update: 7 dashboard failures (and a different error!) 😕

myasonik · 2021-02-19T19:23:22Z

No flaky tests

🎉 🎉 🎉

myasonik · 2021-02-19T19:26:11Z

@elasticmachine merge upstream

myasonik · 2021-02-19T19:43:31Z

test/accessibility/services/a11y/analyze_with_axe.js

+              // EUI bug: https://github.com/elastic/eui/issues/4474
              id: 'aria-required-parent',
-              selector: '[class=*"euiDataGridRowCell"][role="gridcell"] ',
+              selector: '[class=*"euiDataGridRowCell"][role="gridcell"]',
+            },
+            {
+              // 3rd-party library; button has aria-describedby
+              id: 'button-name',
+              selector: '[data-rbd-drag-handle-draggable-id]',
+            },
+            {
+              // EUI bug: https://github.com/elastic/eui/issues/4536
+              id: 'duplicate-id',
+              selector: '.euiSuperDatePicker *',


Ignore all 3rd party errors because there's nothing we can do directly in Kibana to resolve them until they're resolved upstream.

myasonik · 2021-02-19T19:48:38Z

x-pack/test/accessibility/apps/dashboard_edit_panel.ts

-      await testSubjects.click('embeddablePanelMore-mainMenu');
-      await testSubjects.click('embeddablePanelAction-deletePanel');
+    it('can delete panel', async () => {
+      await dashboardPanelActions.removePanel();


Use team-maintained page objects if available because sometimes "removing a panel" is more complicated than it seems

myasonik · 2021-02-19T19:49:28Z

x-pack/test/accessibility/apps/uptime.ts

@@ -61,7 +60,11 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {

    it('overview alert popover controls', async () => {
      await uptimeService.overview.openAlertsPopover();
+      await toasts.dismissAllToasts();


Whenever you see an error like #90555 it's probably a toasts issue, even if you're actively opening/closing popovers/flyouts.

myasonik · 2021-02-19T19:51:25Z

x-pack/test/accessibility/apps/uptime.ts

      await a11y.testAppSnapshot();
+    });
+
+    it('overview alert popover controls nested content', async () => {


Each snapshot should go into its own test just to make identifying the point of failure easier.

The test name is always easy to find but figuring out where within a test you are can be difficult and axe is basically a black box.

dominiqueclarke

Looks good to me from a code perspective. I'm still new but I don't suppose I have to pull down this branch to test since the functional tests, which is what we are changing, are tested against the CI.

dominiqueclarke · 2021-02-19T19:59:26Z

x-pack/test/accessibility/apps/uptime.ts

@@ -61,7 +60,11 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {

    it('overview alert popover controls', async () => {
      await uptimeService.overview.openAlertsPopover();
+      await toasts.dismissAllToasts();


myasonik · 2021-02-19T20:50:04Z

@elasticmachine merge upstream

kibanamachine · 2021-02-20T00:13:17Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💔 Build #108165 failed 4804a0dcea29dcf3418472bd56b9acf8e0be7437
💔 Build #108147 failed f4a6e256bbb40d97a717853ad00c49380505915c
💔 Build #108116 failed 30b6cf7
💔 Build #107849 failed c9084c0
💚 Build #107807 succeeded 04c1e7d5e63bf3744dae7a3ebcbada41f59efd16

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2021-02-20T00:23:35Z

💔 Backport failed

❌ 7.x: Commit could not be cherrypicked due to conflicts

To backport manually, check out the target branch and run:
node scripts/backport --pr 91017

…ndition-for-hiding-recommded-allocation * 'master' of github.com:elastic/kibana: (117 commits) [coverage] ingest data in parallel (elastic#92074) [Lens] Drag and drop performance improvements (elastic#91641) A few more environment uiFilters fixes (elastic#92044) Enabling Uptime and Dashboard a11y test (elastic#91017) [Security Solution][Detections] Adds more granular validation for nested fields (elastic#92041) [Security Solution] [Detections] add overflow-wrap for description (elastic#91945) [Security Solution] [Detections] do not truncate filename in value list table in modal (elastic#91952) Skip flaky apm test elastic#91673 (elastic#92065) [docker] Default server.name to hostname (elastic#90799) Use documentation link service for snapshot restore (elastic#91596) [Security Solution] Clearing up all jest errors and warnings (elastic#91740) Add `@kbn/analytics` to UI Shared Deps (elastic#91810) [7.12][Telemetry] Add missing fields for security telemetry (elastic#91920) [Security Solution] Adds cypress-pipe (elastic#91550) [ML] Fix event rate chart annotation position (elastic#91899) [APM] Break down error table api removing the sparklines (elastic#89138) docs: update dependencies table bug (elastic#91964) [Time to Visualize] Stay in Edit Mode After Dashboard Quicksave (elastic#91729) Unskip Search Sessions Management UI test (elastic#90110) [Fleet] Handle long text in agent details page (elastic#91776) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/__jest__/client_integration/edit_policy/edit_policy.helpers.tsx # x-pack/plugins/index_lifecycle_management/__jest__/components/edit_policy.test.tsx

…bana into task-manager/docs-monitoring * 'task-manager/docs-monitoring' of github.com:gmmorris/kibana: (29 commits) Update docs/developer/plugin-list.asciidoc Update docs/api/task-manager/health.asciidoc Update docs/api/task-manager/health.asciidoc [Lens] Load indexpatterns list from indexPattern Service (elastic#91984) [coverage] ingest data in parallel (elastic#92074) [Lens] Drag and drop performance improvements (elastic#91641) A few more environment uiFilters fixes (elastic#92044) Enabling Uptime and Dashboard a11y test (elastic#91017) [Security Solution][Detections] Adds more granular validation for nested fields (elastic#92041) [Security Solution] [Detections] add overflow-wrap for description (elastic#91945) [Security Solution] [Detections] do not truncate filename in value list table in modal (elastic#91952) Skip flaky apm test elastic#91673 (elastic#92065) [docker] Default server.name to hostname (elastic#90799) Use documentation link service for snapshot restore (elastic#91596) [Security Solution] Clearing up all jest errors and warnings (elastic#91740) Add `@kbn/analytics` to UI Shared Deps (elastic#91810) [7.12][Telemetry] Add missing fields for security telemetry (elastic#91920) [Security Solution] Adds cypress-pipe (elastic#91550) [ML] Fix event rate chart annotation position (elastic#91899) [APM] Break down error table api removing the sparklines (elastic#89138) ...

* master: (36 commits) [Uptime] Thumbnail full screen view steps navigation fix (elastic#91895) Implement ScopedHistory.block (elastic#91099) [Lens] Fix overlowing content on a chart for charts and table (elastic#92006) handle source column differences in embeddable as well (elastic#91987) [Vega] [Map] disable map rotation using right right click / touch rotation gesture (elastic#91996) [Lens] Load indexpatterns list from indexPattern Service (elastic#91984) [coverage] ingest data in parallel (elastic#92074) [Lens] Drag and drop performance improvements (elastic#91641) A few more environment uiFilters fixes (elastic#92044) Enabling Uptime and Dashboard a11y test (elastic#91017) [Security Solution][Detections] Adds more granular validation for nested fields (elastic#92041) [Security Solution] [Detections] add overflow-wrap for description (elastic#91945) [Security Solution] [Detections] do not truncate filename in value list table in modal (elastic#91952) Skip flaky apm test elastic#91673 (elastic#92065) [docker] Default server.name to hostname (elastic#90799) Use documentation link service for snapshot restore (elastic#91596) [Security Solution] Clearing up all jest errors and warnings (elastic#91740) Add `@kbn/analytics` to UI Shared Deps (elastic#91810) [7.12][Telemetry] Add missing fields for security telemetry (elastic#91920) [Security Solution] Adds cypress-pipe (elastic#91550) ...

kibanamachine · 2021-02-23T00:27:08Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 91017 or prevent reminders by adding the backport:skip label.

myasonik · 2021-02-23T01:42:51Z

Manually backported: #92325

# Conflicts: # x-pack/test/accessibility/apps/uptime.ts

Michail Yasonik added 3 commits February 10, 2021 15:00

enable uptime a11y test

953ada5

Merge remote-tracking branch 'upstream/master' into 90555

8639776

fixing a11y tests

f3a711b

myasonik changed the title ~~Enabling Uptime a11y test~~ Enabling Uptime and Dashboard a11y test Feb 17, 2021

remove testing function

c153e93

myasonik added release_note:skip Skip the PR/issue when compiling release notes v7.13.0 v8.0.0 labels Feb 18, 2021

fixing flakiness

df7dc21

fix dashboard tests

c9084c0

myasonik force-pushed the 90555 branch from 04c1e7d to c9084c0 Compare February 18, 2021 21:21

swallow error messages

30b6cf7

myasonik commented Feb 19, 2021

View reviewed changes

dominiqueclarke approved these changes Feb 19, 2021

View reviewed changes

Michail Yasonik added 2 commits February 19, 2021 16:57

Merge remote-tracking branch 'upstream/master' into 90555

bc62d98

revert common_page object toasts handling

572b2fc

myasonik force-pushed the 90555 branch from 4804a0d to 572b2fc Compare February 19, 2021 21:58

myasonik added the auto-backport Deprecated - use backport:version if exact versions are needed label Feb 19, 2021

myasonik merged commit d23c01b into elastic:master Feb 20, 2021

myasonik deleted the 90555 branch February 20, 2021 00:21

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Feb 23, 2021

myasonik mentioned this pull request Feb 23, 2021

[7.x] Enabling Uptime and Dashboard a11y test (#91017) #92325

Merged

myasonik added backported and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Feb 23, 2021

myasonik pushed a commit that referenced this pull request Feb 23, 2021

Enabling Uptime and Dashboard a11y test (#91017) (#92325)

41fa5f6

# Conflicts: # x-pack/test/accessibility/apps/uptime.ts

peteharverson mentioned this pull request Mar 2, 2021

[ML] Transforms: Adds a11y tests for Transform plugin #93197

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enabling Uptime and Dashboard a11y test #91017

Enabling Uptime and Dashboard a11y test #91017

myasonik commented Feb 10, 2021 •

edited

Loading

myasonik commented Feb 18, 2021

myasonik commented Feb 18, 2021

myasonik commented Feb 18, 2021

myasonik commented Feb 19, 2021

myasonik commented Feb 19, 2021 •

edited

Loading

myasonik commented Feb 19, 2021

myasonik Feb 19, 2021

myasonik Feb 19, 2021

myasonik Feb 19, 2021

dominiqueclarke Feb 19, 2021

myasonik Feb 19, 2021

dominiqueclarke left a comment

dominiqueclarke Feb 19, 2021

myasonik commented Feb 19, 2021

kibanamachine commented Feb 20, 2021

kibanamachine commented Feb 20, 2021

kibanamachine commented Feb 23, 2021

myasonik commented Feb 23, 2021

Enabling Uptime and Dashboard a11y test #91017

Enabling Uptime and Dashboard a11y test #91017

Conversation

myasonik commented Feb 10, 2021 • edited Loading

myasonik commented Feb 18, 2021

myasonik commented Feb 18, 2021

myasonik commented Feb 18, 2021

myasonik commented Feb 19, 2021

myasonik commented Feb 19, 2021 • edited Loading

myasonik commented Feb 19, 2021

myasonik Feb 19, 2021

Choose a reason for hiding this comment

myasonik Feb 19, 2021

Choose a reason for hiding this comment

myasonik Feb 19, 2021

Choose a reason for hiding this comment

dominiqueclarke Feb 19, 2021

Choose a reason for hiding this comment

myasonik Feb 19, 2021

Choose a reason for hiding this comment

dominiqueclarke left a comment

Choose a reason for hiding this comment

dominiqueclarke Feb 19, 2021

Choose a reason for hiding this comment

myasonik commented Feb 19, 2021

kibanamachine commented Feb 20, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Feb 20, 2021

💔 Backport failed

kibanamachine commented Feb 23, 2021

myasonik commented Feb 23, 2021

myasonik commented Feb 10, 2021 •

edited

Loading

myasonik commented Feb 19, 2021 •

edited

Loading