Read only window.getComputedStyle() equivalent for accessible state

[Westbrook]

First a clarifying question:

In issues like #60, the word accessibleNode is sometimes used in reference to properties like el.role and other times to properties like el.accessibleNode.role, it's been a while since many of these issues but did conversations like those mostly converge on AriaMixin which has made its way to browsers, or are they referencing still open discussions?

If they are still open, I think this might add to the discussion. If they are not, this new discussion might be able to learn from any roadblocks that conversation ran into.

Looking at issues like #203 and #197, there seems to be some promising motion towards better testability of accessibility when you have some level of browser integration at test time. However, there are many tools that are not, and/or not able to, being built with such an integration. See for instance the industry standard (to the best of my knowledge) testing tool axe-core. Existing over the top of the browser is DOM scope, the things that it can see are much less than a tool integrated with the browser. This plays specifically into the el.setAttribute('role', A) vs el.role = B conversation and then is exacerbated by the complexities of el.#elementInernals.role = C, et al. How can we clearly surface accessible data about a DOM node with so many paths to apply these values and not all of them "reflecting" in some way?

It could be said that the DOM scope is not the place to be testing these values. However, I would point to the difficulty and expense of integration and E2E testing, let alone manual testing, as a major reason why developers look to be able to test accessibility at this level and if we unable to surface this data at that level we're likely to get more comments like this wherein tools makers point to not using modern APIs for their lack of ability to extra the current accessible state when applying it in that way.

With these things in mind, I'd like to propose that we work towards the addition of an equivalent to the window.getComputedStyle() API that could surface the computed value of these important properties.

window.getComputedAria() vs window.getComputedAccessibilityNode() vs alernatives

window.getComputedAria() has some implications that it really just surfaces a readonly object reflecting the AriaMixin, which doesn't technically surface all of the accessibility data in relation to a node. That doesn't mean it couldn't surface more data that currently available on AriaMixin, but any confusion that was caused by that intersection would be a self inflicted wound that we can certainly avoid.

window.getComputedAccessibilityNode() would seem to have room for the larger amount of accessibility data. However, it sounds a lot more like the Accessibility Tree, where in much of the values that we set via built-in elements, attributes, properties, et al seemingly get flattened and simplified for transport to screen readers. See the output of APIs like https://playwright.dev/docs/api/class-accessibility#accessibility-snapshot which end up with less data than was applied to the elements, though maybe focused on the important data?

Another name would certainly be bike shed for this API from the window. I think that coming from the window has some value here in that the API adds no weight to the element itself and implies an out of band usage, like testing, rather than something that should practically be leveraged in live applications. The implied el.accessibleNode. API in other issues could seemingly cover this as well, but it's unclear whether there's agreement to what that surfaces and the idea that it may be a read/write surface means that it might not be as reliable as an over the top API.

Returns

The data this sort of API surfaces, separate of what is available in AriaMixin would be a big question. The data being "computed" would point to wanting something like what we see in Chrome's DevTools:

Which is different than what is surfaced in Firefox:

I'm unsure whether there's a WebKit corollary.

Hopefully tools developers like @WilcoFiers and other will have thoughts on exactly what would be useful to them in this area.

[WilcoFiers]

Thanks for opening this issue @Westbrook. I think this could be a very useful API for developers. For tools like axe-core this doesn't solve the problem with ElementInternals though. It's different enough that I think it requires a separate issue for discussion.

[Westbrook]

@WilcoFiers how do you see this as different? I would love to see an issue with your concerns more fully fleshed out!

From my vantage point, whether it's set via attribute <div role="..."> or on the element el.role="..." or on the internals el.internals_.role="...", the only thing that should matter is the final computed value, not where that value may have come from.

[WilcoFiers]

@Westbrook Sure, so if for example someone puts internals.role = 'buton', that's not going to end up as a computed property. In order to find that axe-core (and other tools) need access to the internals object.

More generally, only knowing that something is wrong isn't enough. Testers (axe, other tools, human auditors) need to be able to report where a problem occurred. Its much easier for a dev to fix an issue if I can tell them if the problem is with the role attribute or with the element's internals. We need to give insight on why something is wrong, not just that it's wrong. Browser devtools do this a lot too. You don't just get the computed CSS, you get all CSS rules that apply to an element, in the order they apply, and whether that was inherited or not. This information helps find and fix problems. Chrome devtools does the same for accessible names. It tells you what attribute a name comes from, and the order in which they are applies.

ElementInternals is especially problematic. Its a technology almost nobody knows about, you can't see that an element has internals in any browser's devtools (or at least, I couldn't find it). Accessibility tools like axe are going to need to explain where this property that can't be seen in the DOM comes from. Otherwise everyone's going to treat it as a false positive and ignore it.

That's one issue. The other is that axe-core (and most tools like it) try to be consistent across browsers. There is no standard for the accessibility tree. Chrome, Safari and Firefox all have their differences, and they change somewhat regularly. If something is a problem in Firefox, we'll want to report that even if someone is testing in Chrome (or vice versa). Axe can do that today because it computes roles / states itself. It also works across different versions of a browser. Chrome auto-updating shouldn't change your accessibility score. In order to continue that strategy axe needs the internals object, or at least something that can very directly tell it what's on there, errors and all.

That's both a matter of convenience (not having to run in more than one browser), and building trust in the tool. There's nothing that'll get developers to ignore an accessibility issue more quickly than learning that you only get the error in one browser.

[Westbrook]

Thanks for that information, that clarifies a lot for me! Am I understanding correctly that there are two major questions here:

1. Does an API in this area surface Element Internals originating values?
2. How does a testing context report how a positive or negative result came into being?

Please let me know if I have over simplified.

---

Does an API in this area surface Element Internals originating values?

Short answer, I think is should.

If in JS the role is set via el.internals.role = 'button' then the primary success indicator of this sort of API for me is that calling window.getComputedAccessibilityNode(el) will return something that includes:

AccessibilityNode {
    // ... information
    // ... information
    role: 'button'
}

If it didn't, it wouldn't be additive to the APIs already available in the browser.

Similarly, were the same el.internals.role = 'button' to be given el.setAttribute('role', 'link') then an ensuing call to window.getComputedAccessibilityNode(el) should return:

AccessibilityNode {
    // ... information
    // ... information
    role: 'link'
}

So much like winodw.getComputedStyle(el) the results would be the actual value of a thing, regardless of how it came to be applied to the element.

How does a testing context report how a positive or negative result came into being?

This one is harder and different, and it is super helpful for you to raise it.

Hopefully someone smarter than me has thoughts here, but I'm not sure how one would go about doing that 100%. Today, how does axe-core do similarly for native elements that report differently between between browsers and screen readers? <search> is widely supported in evergreen browsers, but it reports differently to screen readers by browser, is that a close enough comparison to learn about?

For knowledge, how does axe-core test the accessibility of a list <ul><li></li><ul>? Is it doing it simply by convention, <ul> and <li> elements act this way so it's OK for them to be related as such? If assumption by convention is enough in some cases, it feels like the assumption that by convention if there is no role attribute (el.hasAttribute('role') === false) and no role property (el.role === null) then anything not currently tracked by convention would have to have had this value set via Element Internals? It's not a "fix this at line 7 of your JS" sort of result, but definitely a directional piece of feedback.

Beyond that, it may be that a "corrective" API requires deeper integration with the browser, whether that be via DevTools Protocols, a specifically instrumented-for-testing browser, or something else. Again, hopefully smarter people than I...

[alice]

I can add a bit more context here, I think.

#60 is definitely obsolete, for a start; I've just closed it to reflect that. https://github.com/WICG/aom/blob/gh-pages/explainer.md#what-happened-to-accessiblenode gives a brief answer as to why we moved away from accessibleNode as originally conceived.

James' update there pointing to #197 alludes to conversations that have happened elsewhere (where exactly that was, I unfortunately have absolutely no memory; maybe others do) about the importance of any "computed accessibility properties" API being only available for use in testing contexts.

Essentially, a getComputedAccessibility() API would be like getComputedStyle() in terms of creating a performance bottleneck, but even more so since it would require forcing a clean layout and potentially recursive computation of accessibility properties. Plus, while getComputedStyle() is reading a data structure that needs to be created for normal usage of a web page (even for AT users!), getComputedAccessibility() is reading a structure which is not computed for most users and which is expensive to keep live. This means that either getComputedAccessibility() would need to trigger building a live accessibility tree, which may never be read again in the lifetime of the page but would need to keep being updated, potentially making the page unusably slow [1]; or, each call to getComputedAccessibility() would need to build as much of the full tree as necessary to compute all of the accessibility properties for a Node/Element, each time it was called.

All that being the case, we didn't want to risk an API which as far as we could tell only had real uses for testing (where the performance downsides are largely negligible, or at least straightforward to work around) coming to be used for anything end-user-facing.

In those discussions, we were definitely sympathetic to the use case of tools like axe-core, and the bind DOM-based test tools get in when we have ways to modify the computed accessibility tree that can't be read back from the DOM. However, we just couldn't think of a way to make a web-facing API which nevertheless could only be accessed in testing contexts.

---

[1] Yes, this performance problem can be a big problem for AT users.

[cookiecrook]

@alice wrote:

@cookiecrook's update there pointing to #197 alludes to conversations that have happened elsewhere (where exactly that was, I unfortunately have absolutely no memory; maybe others do) about the importance of any "computed accessibility properties" API being only available for use in testing contexts.

There were a few reasons I recall. The performance impact you mentioned earlier was one of them, and I'm not sure I recall all the others, but here are a few.

1. There was some concern that an author opt-in might open previously unexplored attack vectors for malicious actors looking for technical exploits in code paths that are not otherwise executed on most machines.
2. The fact that some engines don't actively turn off the accessibility runtime after use, so allowing a web author to opt-in to using this may affect the performance of other sites open during the same session or in the forward/back history of the same tab... Though perhaps this will be less of an issue as additional parts of the stack are compartmentalized for various reasons.
3. There was some concern that an author ability to opt-in may result in inadvertent leaking of AT usage or more specific usage patterns, though I believe most of the potential detectability pitfalls I recall mentioned already occur today. Perhaps any new pitfalls could be mitigated with engine/browser/AT updates, except perhaps any fingerprinting ability used to target vulnerable communities with social networking and fraud.

In the meantime, having the API be test-only provided reliability benefit while deferring the risks of a more expansive API.