chore: remove unnecessary axe-config rules #6449
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
katydecorah
approved these changes
Feb 22, 2023
dbjorge
added a commit
to microsoft/accessibility-insights-service
that referenced
this pull request
Feb 22, 2023
…pty-table-header rule) (#2402) #### Details Today, Service decides which axe-core rules to use via a `RuleExclusion` mechanism that starts from axe-core's default ruleset an disables rules we don't use in web. This is inconsistent with how accessibility-insights-web defines its ruleset (via a combination of [axe-core's rule tags and explicit overrides](https://github.com/microsoft/accessibility-insights-web/blob/[email protected]/src/scanner/get-rule-inclusions.ts)). The difference in mechanisms led to a mistake during the axe-core 4.6.3 update where the rulesets became inconsistent between web and service. To ensure consistency between web and service, this PR replaces `rule-exclusion.ts` with a new mechanism based on the recently-added `@accessibility-insights/axe-config` package in the web repo. That package builds an axe config blob based on web's `get-rule-inclusions.ts` mechanism. This update bundles a copy of the current output of that package into `axe-configuration.ts` to form the basis of service's axe configuration, and removes `rule-exclusion.ts` entirely as obsolete. Besides keeping the rules in sync, this also means adds a new axe-core config option that was previously overridden by web but not service (`"pingWaitTime": 1000` instead of the default 500ms). I decided to keep the new value from the web config for consistency with web and because both values are still very comfortably within the margin of error set by the page level timeouts used in `page-timeout-config.ts`, even assuming pages with multiple levels of iframe nesting. ##### Motivation * Make it more reliable to keep web and service rules in sync. * Disables the unintentionally-enabled `empty-table-header` best-practice rule ##### Context * Builds on @sfoslund 's work in microsoft/accessibility-insights-web#6395 * Partially implements (but doesn't complete) #571 * I considered doing a fuller implementation that would involve creating a publishing pipeline for the axe-config package, but I didn't want to tie up the fix for the rule inconsistency in service vs web with that process. This is still an incremental improvement that moves us closer to that end state without the NPM package layer. * Incorporates changes from microsoft/accessibility-insights-web#6448 and microsoft/accessibility-insights-web#6449 ##### Testing Besides the PR checklist, I validated that a local build of the scan package against test site https://www.catalog.update.microsoft.com/Home.aspx produced a report consistent with [email protected]'s results (no violations), where previously the service reported a slightly distinct set of errors (one violation of the `empty-table-header` rule, which was accidentally omitted from `rule-exclusions.ts` during the 4.6.3 update). #### Pull request checklist <!-- If a checklist item is not applicable to this change, write "n/a" in the checkbox --> - [x] Addresses an existing issue: Partially addresses #571 - [x] Added relevant unit test for your changes. (`yarn test`) - [x] Verified code coverage for the changes made. Check coverage report at: `<rootDir>/test-results/unit/coverage` - [x] Ran precheckin (`yarn precheckin`) - [ ] Validated in an Azure resource group
|
The failures here are legitimate; it turns out that removing an automated checks rule from assessment breaks backwards compatibility with old I think the ideal here would be to update the load assessment path with a means of allowing for silently dropping passing automated checks that were removed. But in the interest of timeboxing this cleanup PR, I'm going to drop the removal of the bypass rule and leave that better change for whatever future feature work actually needs it. |
katydecorah
approved these changes
Mar 1, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Details
This PR:
Adds an exclusion for thebypassrule, which axe-core changed in 4.2.0 to be a "reviewOnFail" rule (ie, it only ever returns "pass" or "needs review"), but which we were still running anyway and just never reporting any results for (since we don't have needs review content for bypass). Service already excluded this rule explicitly.@accessibility-insights/axe-configpackage to not include an entry forframe-tested. We include this rule in AI4Web's scans, but then post-process the results such that we don't present it to users as an automated checks failure (we instead present it as a special warning bar). This change means that a user relying on axe-config won't includeframe-tested, which will avoid the appearance of issues reported byaxe-configbut not by automated checks.Motivation
marginal perf improvement by not running unnecessary ruleContext
n/a
Pull request checklist
yarn fastpassyarn test)<rootDir>/test-results/unit/coveragefix:,chore:,feat(feature-name):,refactor:). SeeCONTRIBUTING.md.