Add pa11y-ci for automated accessibility tests with htmlcs runner #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
julien-deramond
force-pushed
the
main-jd-pa11y-ci-htmlcs
branch
from
3a06d53 to
ffa7da8
Compare
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This Draft PR is originated from both twbs#29315 and Boosted use of pa11y / pa11y-ci
Run
Local
CLI mode
JSON mode
Focus on a specific page
GitHub CI
At the end, you can retrieve the HTML report:
Then, open the
index.htmlfile in the .zip file:Topics to tackle
Find a way to use the HTML reporter but still log the results in CLI
It seems possible in the
masterbranch ofpa11y-civia pa11y/pa11y-ci@5c842cf that allows multiple reporters.We should be able to do something like:
{ "defaults": { "reporters": [ "cli", "pa11y-reporter-html" // or "json" maybe ] },But we would have to wait for the 2.4.3 or 2.5.0 to be released.
For the moment, the CLI version is only available locally.
pa11y-ci crashes
XhmikosR: "due to our JS using ES6+, the puppeteer version pa11y-ci depends on is currently crashing. :/ We need them to cut a new pa11ci release with the update pa11y package at version 6."
See if we can use npx maybe, since Chromium is a huge download or see if we can cache puppeteer on CI
XhmikosR: "I don't want the huge puppeteer (which in turn downloads Chromium) devDependency, ~143MB... Not sure how to work around this, we could probably move the needed packages in the Action that needs them, but it's not a perfect solution either"
Which runner?
axelike Boosted?htmlcslike Bootstrap POChtmlcsrunner: Analyze and fix the errors or usehideElementswhen necessaryTests of the 21st of October
Total without any
hideElements: 568 errorshideElementsrule<form class="bd-search position-relative me-auto"> <span class="algolia-aut...</form>.bd-search<li class="mb-2">Designed and built with all the...</li>footer > div > div > div:nth-child(1) > ul > li<a href="https://getbootstrap.com/">Bootstrap</a>footer > p > a<strong class="text-success">Correct</strong>div:nth-child(4) > main > div:nth-child(3) > div:nth-child(9) > div > strong<button type="button" class="btn btn-outline-warning">Warning</button>button.btn.btn-outline-warning, button.btn.btn-outline-info, button.btn.btn-outline-light<button class="btn btn-outline-success" type="submit">Search</button>nav.navbar-dark.bg-dark button.btn.btn-outline-success<h2 class="h6 pt-4 pb-3 mb-4 border-bottom">On this page</h2>aside > h2<fieldset disabled="" aria-label="Disabled fieldset example"> <div class="mb-3">...</fieldset>fieldset *:not(legend)doesn't seem to work so#disabled-forms > div:nth-child(2) > div > form > fieldset<form> <div class="form-flo...</form>form:not(:has(> [type=submit]))doesn't work so#floating-labels > div:nth-child(2) > div > form<a class="nav-link" href="#scrollspyHeading2">Second</a>.navbar.navbar-light > .nav.nav-pills > li > a.nav-link<h6 class="my-0">Promo code</h6>.bg-light > .text-success<a class="dropdown-item disabled">Disabled link</a>ul.dropdown-menu > li > a.dropdown-item.disabled<a href="#" class="link-warning">Warning link</a>(same thing forlink-infoandlink-light) (see outline buttons)a.link-warning, a.link-info, a.link-light<button class="btn btn-outline-success" type="submit">Search</button>(same forbtn-outline-secondaryandbtn-outline-primary)nav.navbar.navbar-light.bg-light .btn.btn-outline-success, nav.navbar.navbar-light.bg-light .btn.btn-outline-secondary, nav.navbar.navbar-light .btn.btn-outline-primary<h4 id="constructor">constructor<a class="anchorjs-l...</h4>(I chose one to hide)h4#constructor<small class="d-block text-muted"> <svg class="bi me-1" w...</small>label.bg-light .text-muted<a class="nav-link ms-3 my-1" href="#item-1-1">Item 1-1</a>nav.navbar-light.bg-light nav.nav.nav-pills > a.nav-link<span>Saved reports</span>.sidebar.bg-light .text-muted<form class="form-floating"> <input type="email" class="f...</form>form.form-floating<button class="btn btn-outline-secondary" type="button">Example button</button>.bg-light .btn.btn-outline-secondary<form> <input class="form-c...</form>div[id^=navbarsExample] > form<a href="/" class="d-flex align-items-center col-md-3 mb-2 mb-md-0 text-dark text-decoration-none"> <svg class="bi me-2" w...</a>main > header > div > div > a, main > div > header > a, main header > div > div > div > a<input type="text" class="form-control" placeholder="Promo code">input.form-control[placeholder='Promo code'], input.form-control[placeholder='رمز ترويجي']<span class="text-muted">(Optional)</span>.bg-light form small.text-muted, .bg-light form span.text-muted<p class="mb-1">© 2017–2021 Company Name</p>.bg-light footer.text-muted > p<a href="#">Privacy</a>.bg-light footer > ul > li > a<form class="p-2 mb-2 bg-light border-bottom"> <input type="search" cla...</form>#dropdownFilter > div > form<button class="btn cal-btn" type="button"> <svg class="bi" widt...</button>and other elements of#dropdownCalendar#dropdownCalendar > div > div > div > select, #dropdownCalendar > div > div > div > button<a>in footers<a href="/" class="col-md-4 d-flex align-items-center justify-content-center mb-3 mb-md-0 me-md-auto link-dark text-decoration-none"> <svg class="bi me-2" wid...</a>footer > p + a[href='/'], footer > div > a[href='/'], footer ul > li > a[href='#']<form> <h5>Subscribe to our...</form>div:nth-child(10) > footer > div:nth-child(1) > div:nth-child(4) > form<p class="text-warning"> No grid classes are nece...</p>p.text-warning<a href="#" class="nav-link px-2 text-secondary">Home</aheader div.bg-dark a.text-secondary, header.bg-dark a.text-secondary<form class="col-12 col-lg-auto mb-3 mb-lg-0 me-lg-3"> <input type="search"...</form>header:nth-child(8) > div > div form, header:nth-child(10) > div > div form, header:nth-child(12) > div > div form, header:nth-child(16) > div form, header:nth-child(18) > div > div form<small class="text-muted">By clicking Sign up, you agree ...</small>form.bg-light small.text-muted<p class="fs-5 mb-4">Quickly design and customize re...</p>div.bg-dark.text-secondary p<code>.container</code>.bg-light code<a href="/docs/5.1/examples/navbar-static/">top</a>.bg-light a<span class="text-muted">Place sticky footer content her...</span>footer.bg-light span.text-muted<form class="container-fluid"> <div class="input-group"> ...</form>main > div:nth-child(3) > div:nth-child(55) > nav > form, main > div:nth-child(3) > div:nth-child(59) > nav > form, main > div:nth-child(3) > div:nth-child(133) > nav > div > button<div class="opacity-100 p-3 m-2 bg-primary text-light fw-bold rounded">100%</div>.opacity-100.bg-primary.text-light, .opacity-75.bg-primary.text-light, .opacity-50.bg-primary.text-light, .opacity-25.bg-primary.text-light<table class="table text-center"> <thead> <tr>...</table>main > div:nth-child(3) > table.table.text-center<p class="mt-5 mb-3 text-muted">© 2017–2021</p>body > main > form > p.text-muted<form> <div class="mb-3"> <labe...</form>main > div:nth-child(3) > div:nth-child(49) > form