Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add aXe testing to automated CI tests #513

Open
nolanlawson opened this issue Aug 28, 2018 · 8 comments
Open

Add aXe testing to automated CI tests #513

nolanlawson opened this issue Aug 28, 2018 · 8 comments
Labels
accessibility Accessibility (a11y) tests

Comments

@nolanlawson
Copy link
Owner

We can use this testcafe plugin: https://github.com/helen-dikareva/axe-testcafe

@nolanlawson nolanlawson added accessibility Accessibility (a11y) chore Test or deployment infrastructure issues tests and removed chore Test or deployment infrastructure issues labels Aug 28, 2018
@sorin-davidoi
Copy link
Contributor

The plugin uses an older version of aXe, but there are two pull requests opened if the latest version is needed.

@nolanlawson
Copy link
Owner Author

Interestingly it seems aXe cannot test Pinafore. I've tried axe-cli as well, and it just hangs.
screenshot from 2018-12-05 22-53-07

I thought maybe it was due to CSP, but I disabled it and it still fails.

On the bright side, Lighthouse does seem to be working so perhaps this should be closed in favor of #746

@sorin-davidoi
Copy link
Contributor

aXe runs for me in both Firefox Nightly and Chromium.

image

@nolanlawson
Copy link
Owner Author

axe works for me now, although it's finding some false positives: dequelabs/axe-core#1176 (comment)

@nolanlawson nolanlawson changed the title Add aXe testing to automated Ci tests Add aXe testing to automated CI tests Oct 16, 2019
@NickColley
Copy link
Collaborator

Want to get this going to ensure when we resolve contrast issues we dont accidentally regress so adding to milestone for WCAG compliance.

@nolanlawson
Copy link
Owner Author

@NickColley Thanks a lot. AA sounds totally achievable for us. (My understanding is that AAA is more like a "stretch goal" so not necessarily something we need to target?)

@NickColley
Copy link
Collaborator

Yeah some things you want to try and aim for AAA but sometimes it's actually impossible to meet all to AAA.

AA is what the main legislation for the EU and UK Public Sector Accessibility Regulations align with, and is becoming a sort of defacto baseline, and definitely achievable for us.

@NickColley
Copy link
Collaborator

Just a note on false positives, it's pretty common for axe to throw up issues that are false positives so what we'd do in this case is disable the rule that is being thrown and document why. This may also be a good way to introduce the tests as well, in a sort of 'failing test' situation where we add axe and raise github issues for each disabled rule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accessibility Accessibility (a11y) tests
Projects
None yet
Development

No branches or pull requests

3 participants