Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: core(axe): new audits empty-table-header, frame-focusable-content #12536

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions lighthouse-cli/test/cli/__snapshots__/index-test.js.snap
Original file line number Diff line number Diff line change
Expand Up @@ -255,9 +255,15 @@ Object {
Object {
"path": "accessibility/duplicate-id-aria",
},
Object {
"path": "accessibility/empty-table-header",
},
Object {
"path": "accessibility/form-field-multiple-labels",
},
Object {
"path": "accessibility/frame-focusable-content",
},
Object {
"path": "accessibility/frame-title",
},
Expand Down Expand Up @@ -594,11 +600,21 @@ Object {
"id": "duplicate-id-aria",
"weight": 10,
},
Object {
"group": "a11y-tables-lists",
"id": "empty-table-header",
"weight": 2,
},
Object {
"group": "a11y-names-labels",
"id": "form-field-multiple-labels",
"weight": 2,
},
Object {
"group": "a11y-navigation",
"id": "frame-focusable-content",
"weight": 3,
},
Object {
"group": "a11y-names-labels",
"id": "frame-title",
Expand Down
14 changes: 13 additions & 1 deletion lighthouse-cli/test/fixtures/a11y/a11y_tester.html
Original file line number Diff line number Diff line change
Expand Up @@ -133,14 +133,26 @@
<section>
<div id="duplicate-id-aria" class="duplicate-id-aria"></span>
<div id="duplicate-id-aria"></div>
<div aria-labelledby="duplicate-id-aria"></div>
<input type="text" aria-labelledby="duplicate-id-aria"/>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be an appropriate use of aria-labelledby or this now gets flagged by aria-allowed-attr. Nice upgrade for that audit

</section>
<p>empty-table-header</p>
<section>
<table>
<tr>
<th></th>
</tr>
</table>
</section>
<p>form-field-multiple-labels</p>
<section>
<label for="form-field-multiple-labels" id="label1">label1</label>
<label for="form-field-multiple-labels">label2</label>
<input type="checkbox" id="form-field-multiple-labels" />
</section>
<p>frame-focusable-content</p>
<section>
<iframe title="some title" tabindex="-1" src="../dobetterweb/focusable-iframe.html"></iframe>
</section>
<p>frame-title</p>
<section>
<iframe id="frame-title"></iframe>
Expand Down
9 changes: 9 additions & 0 deletions lighthouse-cli/test/fixtures/dobetterweb/dbw_tester.html
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,15 @@ <h2>Do better web tester page</h2>
<button class="small-button">Do something</button>
<button class="small-button">Do something else</button>

<!-- FAIL - frame-focusable-content -->
<iframe tabindex="-1" src="focusable-iframe.html"></iframe>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use srcdoc instead to avoid the 1-line extra file? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my first try, didn't work! Not supported I suppose.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my first try, didn't work! Not supported I suppose.

interesting, are srcdoc iframes a unique origin and so not available in the parent via js (so axe wouldn't be able to see inside)?

<!-- FAIL - empty-table-header -->
<table>
<tr>
<th></th>
</tr>
</table>

<!-- Some websites overwrite the original Error object. The captureJSCallUsage function
relies on the native Error object and prepareStackTrace from V8. When overwriting the stack
property the e.stack inside the gatherer won't be in the correct format
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<a href='/'>Home</a>
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ const expectations = [
},
'selector': 'body > section > div > div#aria-treeitem-name',
'snippet': '<div id="aria-treeitem-name" role="treeitem">',
'explanation': 'Fix any of the following:\n Element does not have text that is visible to screen readers\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n aria-label attribute does not exist or is empty\n Element has no title attribute',
'explanation': 'Fix any of the following:\n Element does not have text that is visible to screen readers\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are just rearranging of the explanation concat order. We could drop them from the expectations...but they were also really easy to c/p. I just did that for now.

'nodeLabel': 'div',
},
},
Expand All @@ -154,7 +154,7 @@ const expectations = [
},
'selector': 'body > section > div#aria-command-name',
'snippet': '<div id="aria-command-name" role="button">',
'explanation': 'Fix any of the following:\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute\n Element does not have text that is visible to screen readers',
'explanation': 'Fix any of the following:\n Element does not have text that is visible to screen readers\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute',
'nodeLabel': 'div',
},
},
Expand All @@ -174,7 +174,7 @@ const expectations = [
},
'selector': 'body > section > div#aria-tooltip-name',
'snippet': '<div id="aria-tooltip-name" role="tooltip">',
'explanation': 'Fix any of the following:\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute\n Element does not have text that is visible to screen readers',
'explanation': 'Fix any of the following:\n Element does not have text that is visible to screen readers\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute',
'nodeLabel': 'div',
},
},
Expand Down Expand Up @@ -239,7 +239,7 @@ const expectations = [
'selector': 'body > section > div#aria-required-attr',
'path': '2,HTML,1,BODY,19,SECTION,0,DIV',
'snippet': '<div id="aria-required-attr" role="checkbox">',
'explanation': 'Fix any of the following:\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute\n Element does not have text that is visible to screen readers',
'explanation': 'Fix any of the following:\n Element does not have text that is visible to screen readers\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute',
'nodeLabel': 'div',
},
},
Expand Down Expand Up @@ -314,7 +314,7 @@ const expectations = [
'type': 'node',
'selector': 'body > section > button#button-name',
'snippet': '<button id="button-name">',
'explanation': 'Fix any of the following:\n Element does not have inner text that is visible to screen readers\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element\'s default semantics were not overridden with role="none" or role="presentation"\n Element has no title attribute',
'explanation': 'Fix any of the following:\n Element does not have inner text that is visible to screen readers\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute\n Element\'s default semantics were not overridden with role="none" or role="presentation"',
'nodeLabel': 'button',
},
},
Expand Down Expand Up @@ -438,7 +438,7 @@ const expectations = [
node: {
'type': 'node',
'selector': 'body > section > input#form-field-multiple-labels',
'path': '2,HTML,1,BODY,43,SECTION,2,INPUT',
'path': '2,HTML,1,BODY,45,SECTION,2,INPUT',
'snippet': '<input type="checkbox" id="form-field-multiple-labels">',
'explanation': 'Fix all of the following:\n Multiple label elements is not widely supported in assistive technologies. Ensure the first label contains all necessary information.',
'nodeLabel': 'input',
Expand All @@ -456,7 +456,7 @@ const expectations = [
'type': 'node',
'selector': 'body > section > iframe#frame-title',
'snippet': '<iframe id="frame-title">',
'explanation': 'Fix any of the following:\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element has no title attribute\n Element\'s default semantics were not overridden with role="none" or role="presentation"',
'explanation': 'Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element\'s default semantics were not overridden with role="none" or role="presentation"',
'nodeLabel': 'iframe',
},
},
Expand All @@ -471,7 +471,7 @@ const expectations = [
node: {
'type': 'node',
'selector': 'body > section > h3',
'path': '2,HTML,1,BODY,47,SECTION,1,H3',
'path': '2,HTML,1,BODY,51,SECTION,1,H3',
'snippet': '<h3>',
'explanation': 'Fix any of the following:\n Heading order invalid',
'nodeLabel': 'sub-sub-header',
Expand Down Expand Up @@ -690,6 +690,27 @@ const expectations = [
],
},
},
'frame-focusable-content': {
score: null,
scoreDisplayMode: 'notApplicable',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot get this audit to fail, no matter what I do. There are failing cases in the PR that introduced the rule (dequelabs/axe-core#2785), but even verbatim copy/pasting them doesn't seem to trigger it. When I step through axe-core, it appears the frameFocusableContentMatches never even gets to see the iframes, only the top-level html element is passed in, so I'm not sure what's going on.

(notably the dbw_tester version is causing a passing audit in sample_v2, but not clear why that one isn't notApplicable as well)

We should probably figure out an actual failure case and/or file a bug on why the axe test can fail but we can't (it's late, maybe I'm just missing something), but in the meantime, should we not include in the LH 8 release?

},
'empty-table-header': {
score: null,
details: {
items: [
{
node: {
'type': 'node',
'selector': 'table > tbody > tr > th',
'path': '2,HTML,1,BODY,43,SECTION,0,TABLE,0,TBODY,0,TR,0,TH',
'snippet': '<th>',
'explanation': 'Fix any of the following:\n Unable to determine if element has children',
'nodeLabel': 'th',
},
},
],
},
},
},
},
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@
const expectations = [
{
networkRequests: {
// 50 requests made for normal page testing.
// 53 requests made for normal page testing.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+3 for the iframe request per pass

// 6 extra requests made because stylesheets are evicted from the cache by the time DT opens.
// 3 extra requests made to /dobetterweb/clock.appcache
length: 59,
length: 62,
},
artifacts: {
HostFormFactor: 'desktop',
Expand Down Expand Up @@ -381,11 +381,11 @@ const expectations = [
},
'dom-size': {
score: 1,
numericValue: 149,
numericValue: 154,
details: {
items: [
{statistic: 'Total DOM Elements', value: 149},
{statistic: 'Maximum DOM Depth', value: 4},
{statistic: 'Total DOM Elements', value: 154},
{statistic: 'Maximum DOM Depth', value: 5},
{
statistic: 'Maximum Child Elements',
value: 100,
Expand Down Expand Up @@ -415,7 +415,7 @@ const expectations = [
screenshot: {
width: 360,
// Allow for differences in platforms.
height: '1350±20',
height: '1500±20',
data: /^data:image\/jpeg;.{500,}/,
},
nodes: {
Expand Down
44 changes: 44 additions & 0 deletions lighthouse-core/audits/accessibility/empty-table-header.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/**
* @license Copyright 2021 The Lighthouse Authors. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/
'use strict';

/**
* @fileoverview Ensures table headers have discernible text.
* See base class in axe-audit.js for audit() implementation.
*/

const AxeAudit = require('./axe-audit.js');
const i18n = require('../../lib/i18n/i18n.js');

const UIStrings = {
/** Title of an accesibility audit that checks if there are any table headers without visible text. This title is descriptive of the successful state and is shown to users when no user action is required. */
title: '`table` headers have visible text',
/** Title of an accesibility audit that checks if there are any table headers without visible text. This title is descriptive of the failing state and is shown to users when there is a failure that needs to be addressed. */
failureTitle: '`table` headers do not have visible text',
/** Description of a Lighthouse audit that tells the user *why* they should try to pass. This is displayed after a user expands the section to see more. No character length limits. 'Learn More' becomes link text to additional documentation. */
description: '`table` headers should have visible text so assistive technologies like screen readers may describe table contents appropriately. [Learn more](https://dequeuniversity.com/rules/axe/4.2/empty-table-header).',
};

const str_ = i18n.createMessageInstanceIdFn(__filename, UIStrings);

class EmptyTableHeader extends AxeAudit {
/**
* @return {LH.Audit.Meta}
*/
static get meta() {
return {
id: 'empty-table-header',
title: str_(UIStrings.title),
failureTitle: str_(UIStrings.failureTitle),
description: str_(UIStrings.description),
scoreDisplayMode: AxeAudit.SCORING_MODES.INFORMATIVE,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty-table-header only ever passes or is incomplete. Based on #10072, we should be handling those audits as scoreDisplayMode: 'informative', so here we are.

I was out on leave for that PR so I don't have all the background, but today it does seems a little strange to not have it count for anything when passing, even if axe can't be sure the failing case is really failing.

That said, it seems like our a11y_tester example really is a failure, and it's not clear to me why "Unable to determine if element has children" is the best we can do (the <th> is plainly empty) and if it's useful to show that to users.

Maybe for now we should drop this audit after all and re-add it when/if it gains some certainty?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requiredArtifacts: ['Accessibility'],
};
}
}

module.exports = EmptyTableHeader;
module.exports.UIStrings = UIStrings;
43 changes: 43 additions & 0 deletions lighthouse-core/audits/accessibility/frame-focusable-content.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/**
* @license Copyright 2021 The Lighthouse Authors. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/
'use strict';

/**
* @fileoverview No `<iframe tabindex=-1>` has focusable content.
* See base class in axe-audit.js for audit() implementation.
*/

const AxeAudit = require('./axe-audit.js');
const i18n = require('../../lib/i18n/i18n.js');

const UIStrings = {
/** Title of an accesibility audit that checks if there are any iframes with focusable content that mistakenly set tabindex = -1. This title is descriptive of the successful state and is shown to users when no user action is required. */
title: '`<iframe>` elements with focusable content do not have a negative `tabindex`',
/** Title of an accesibility audit that checks if there are any iframes with focusable content that mistakently set tabindex = -1. This title is descriptive of the failing state and is shown to users when there is a failure that needs to be addressed. */
failureTitle: '`<iframe>` elements with focusable content have a negative `tabindex`',
/** Description of a Lighthouse audit that tells the user *why* they should try to pass. This is displayed after a user expands the section to see more. No character length limits. 'Learn More' becomes link text to additional documentation. */
description: 'Focusable elements within an `<iframe>` element with a negative `tabindex` cannot be reached via keyboard navigation. [Learn more](https://dequeuniversity.com/rules/axe/4.2/frame-focusable-content).',
};

const str_ = i18n.createMessageInstanceIdFn(__filename, UIStrings);

class FrameFocusableContent extends AxeAudit {
/**
* @return {LH.Audit.Meta}
*/
static get meta() {
return {
id: 'frame-focusable-content',
title: str_(UIStrings.title),
failureTitle: str_(UIStrings.failureTitle),
description: str_(UIStrings.description),
requiredArtifacts: ['Accessibility'],
};
}
}

module.exports = FrameFocusableContent;
module.exports.UIStrings = UIStrings;
4 changes: 4 additions & 0 deletions lighthouse-core/config/default-config.js
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,9 @@ const defaultConfig = {
'accessibility/document-title',
'accessibility/duplicate-id-active',
'accessibility/duplicate-id-aria',
'accessibility/empty-table-header',
'accessibility/form-field-multiple-labels',
'accessibility/frame-focusable-content',
'accessibility/frame-title',
'accessibility/heading-order',
'accessibility/html-has-lang',
Expand Down Expand Up @@ -517,7 +519,9 @@ const defaultConfig = {
{id: 'document-title', weight: 3, group: 'a11y-names-labels'},
{id: 'duplicate-id-active', weight: 3, group: 'a11y-navigation'},
{id: 'duplicate-id-aria', weight: 10, group: 'a11y-aria'},
{id: 'empty-table-header', weight: 2, group: 'a11y-tables-lists'},
{id: 'form-field-multiple-labels', weight: 2, group: 'a11y-names-labels'},
{id: 'frame-focusable-content', weight: 3, group: 'a11y-navigation'},
{id: 'frame-title', weight: 3, group: 'a11y-names-labels'},
{id: 'heading-order', weight: 2, group: 'a11y-navigation'},
{id: 'html-has-lang', weight: 3, group: 'a11y-language'},
Expand Down
3 changes: 3 additions & 0 deletions lighthouse-core/gather/gatherers/accessibility.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ async function runA11yChecks() {
'svg-img-alt': {enabled: false},
'audio-caption': {enabled: false},
'aria-treeitem-name': {enabled: true},
'empty-table-header': {enabled: true},
// https://github.com/dequelabs/axe-core/issues/2958
'nested-interactive': {enabled: false},
},
});

Expand Down
Loading