Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pa11y-ci for automated accessibility tests #29315

Closed
wants to merge 3 commits into from

Conversation

XhmikosR
Copy link
Member

@XhmikosR XhmikosR commented Aug 28, 2019

This is mostly a POC and it will need a lot more tweaking and some of my other PRs finished first, like #29208.

  • Look into ignoring some classes known to fail in hideElements (*-success, etc) ("hideElements": "[class*=-success], [class*=-info]")
  • Maybe ignore examples from color contrast check?
  • See if we can use html reporter (https://github.com/pa11y/pa11y-reporter-html)
  • Find a way to use the HTML reporter but still log the results in CLI
  • See if we can drop delay-cli
  • Upload HTML report on CI
  • See if we can use npx maybe, since Chromium is a huge download or see if we can cache puppeteer on CI

@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch 9 times, most recently from fd1a9dc to 6910e98 Compare August 30, 2019 15:40
@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch from 6910e98 to f0ee28b Compare September 6, 2019 15:26
@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch 5 times, most recently from 4401abe to 601ead7 Compare January 9, 2020 15:25
@XhmikosR
Copy link
Member Author

XhmikosR commented Jan 13, 2020

I've made some progress with this a few days ago. I added the HTML results to CI so one can download them and review them. The pa11-ci tests run only on Node.js 12 so you need to visit this action to get the results.

AFAICT we need to ignore some selectors that we know that will fail color contrast (and maybe all of the examples?), and we can try fixing the remaining issues.

Any help is welcome. :)

@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch 4 times, most recently from f49f19f to cb81dc5 Compare January 17, 2020 12:50
@ffoodd
Copy link
Member

ffoodd commented Jan 31, 2020

FWIW, pa11y has a --wait argument, than can be passed as an argument in CLI or in the json config file. Not tried in CI but will do soon, could allow to remove the delay-cli dep.

Also while working on this, I temporarily disabled contrast checking since there were too many failures (more than 15.000!) by using:

"ignore": [
      "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18"
    ]

Result is much more readable :)

Still working on it but I might have more to share soon!

@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch 2 times, most recently from aa0a889 to a406a32 Compare November 4, 2020 08:37
@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch from a406a32 to 2bbf187 Compare November 8, 2020 18:51
@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch from a02bb10 to 8353840 Compare November 19, 2020 19:40
@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch 2 times, most recently from 9460fe1 to f782eaa Compare December 4, 2020 08:14
@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch from f782eaa to 4ccbfbd Compare December 15, 2020 08:15
@XhmikosR XhmikosR force-pushed the master-xmr-pa11y-ci branch from 4ccbfbd to ee1562b Compare February 15, 2021 21:02
@patrickhlauke
Copy link
Member

Is this PR still worth pursuing at this point, or do we need a fresh take on it?

@XhmikosR
Copy link
Member Author

XhmikosR commented May 4, 2021

There's no need for a fresh take in it, I can rebase it. We need to solve the TODO I have in the OP.

XhmikosR and others added 3 commits September 28, 2021 19:00
Add pa11y-ci-reporter-html and also upload the HTML results to CI if there are failures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants