Epic: ACT Inconsistency Issues

[shunguoy]

The Checker has some remaining inconsistency issues with the ACT test suit. This Epic is to resolve those issues and any potential new issues during the release cycle.

Epic Details

Hill

Users want the Checker to align with industry-standard (ACT) and to have fewer bugs and false positives

Playback Demonstration

• Each sprint replay will use rules showcase template to explain the issue, including user impact, supporting standard(s), what's broken, fix, help, mapping, and test case(s) to show the issue and the fix.
• Review ACT's mapping (rules to Success Criteria (checkpoints) against Checker's and see if Checker has conflicts - where Checker disagrees, and decide if Checker wants to do anything about it. Review recommendations with Subject Matter Experts (SME Connection)
• W3C ACT published and proposed rule - gaps
  • ACT rule status sheet
  • Take discrepancies to W3C's SME Connection to decide what needs to have a Git issue

Metrics

• Number of rules and help files that are fixed and merged. The Engine rules change summary2024 is updated after each new or fixed rule, help, or Engine change.
• Each Checker version deployed to the stores also includes the Release Notes that include the list of fixes and features that users read.
• Improvement displayed in Checker's ACT implementation report
  • Before (as of 7 November 2024)
  • 
  • After

Comms

The team's overall release notes posted to Slack, Office Hours, other developer meetings, etc. will contain the engine changes, especially the new Checker Rule Set deployment dates that will be an integral part of the communications, such as the Rule Set Options in the Checker Settings.

Issues assigned/related to this Epic:

1. fixrule(img_alt_null) Maps to 1.1.1 while ACT expects None #2089
2. fixrule(img_alt_valid) Reports "inapplicable" while ACT 23a2a8 expects "Failed" for Fail Example 2 #2090
3. fixrule(meta_refresh_delay | meta_redirect_optional) Reports "cannot tell" and "passed" while ACT bisz58 expects "Failed" for 3 Fail Example test cases #2091
4. fixrule(element_orientation_unlocked) Reports "inapplicable" while ACT b33eff expects "Failed" #2092

Related issues

6. Closed original Open issues for ACT implementation inconsistencies #2030
7. Opened issue against ACT: Scrollable content [0ssw9k]: Equal Access checker listed as inconsistent because it doesn't report additional AAA criteria #2198

What project/section is the Epic for?

• checker
• ACT Checker implementation report

Definition of Done

• All content has been reviewed for grammar, spelling, style and technical accuracy
• All code committed to main branch
• All unit tests pass (including accessibility)
• Demo works in staging environment
• At least one performance metric / monitor constructed

Lessons learned and observations