No definition of what constitutes an A, AA, or AAA conformance Level #3889
Comments
Helixopp
changed the title
|
From memory, the decision of which level was assigned to each SC relates back to the old concept of Priority from WCAG 1.0 https://www.w3.org/TR/WAI-WEBCONTENT/#priorities but it does indeed seem that this has not been ported as such to the WCAG 2.x documents. I do think that your question partly mixes up different concepts though: the level for an SC has been set, it is what is is. and WCAG is binary ... you either pass or fail an SC, and in order to claim conformance to a particular level, you as developer must address all SCs at that level (and any level underneath that chosen level, i.e. for AA conformance need to fully address all A and AA). there's no vagueness/wiggle room there. However, it's clear that beyond the binary pass/fail, there are always nuances in how bad a failure actually is in the context of a page/site as a whole. That is something that can't be absolutely and unambiguously defined in the spec, as it's so heavily dependent on context. And THAT is usually what agencies/testers try to indicate through their own additional severity rankings. It still doesn't change anything in terms of the binary pass/fail and the need to satisfy all SCs for your chosen conformance level, but is a subjective extra categorisation that testers give solely to help developers prioritise remediation. They still have to remediate everything that fails, but it acknowledges that some fixes will likely be easier or more impactful or important. I don't think that part can ever be made unambiguous and purely objective.
Can't speak to what other shops do, but we (TetraLogical) base our severity levels primarily on the real-world impact a fail has on actual users, regardless of SC level. Having said all that, agree it would be nice to include a bit of high-level rationale about A/AA/AAA in the non-normative explanatory documentation to just give a feel for why SCs were slotted into the different levels. |
|
It was my understanding that Level A was for items that can't be worked around even with accessibility features or assistive technology and that they are barriers even with other common technologies being present. Some items such as contrast were slotted AA because at the time there were ways to use custom stylesheets for instance to change the text color or contrast and for SC 1.4.4 use of screen magnification software to enlarge the text. For other SC like 1.2.3 and 1.2.5 (and others) the SC at A and AA built on each other to provide layers of access to different degrees. AAA items are ones that may not be able to be met in all situations so they had to be moved to AAA. |
|
There is an explanation in the understanding docs here: https://www.w3.org/WAI/WCAG22/Understanding/conformance#levels As Patrick mentioned, it doesn't (and cannot) take into account the context of the issue, so people assigning severity should use other factors. I appreciate is is hard to find, but where would you have looked for it in the first place? Perhaps we can use that information to add a link in a useful place. |
|
To Patrick’s point the only somewhat acceptable explanation is in WCAG 1.0. But since 2.0 isn’t necessarily some extension of 1.0 that isn’t really satisfactory. Plus WCAG 1.0 refers to web documents not websites or web pages:
5. Conformance
This section defines three levels of conformance to this document:
* Conformance Level "A": all Priority 1 checkpoints are satisfied;
* Conformance Level "Double-A": all Priority 1 and 2 checkpoints are satisfied;
* Conformance Level "Triple-A": all Priority 1, 2, and 3 checkpoints are satisfied;
4. Priorities
Each checkpoint has a priority level assigned by the Working Group based on the checkpoint's impact on accessibility.
[Priority 1]
A Web content developer must satisfy this checkpoint. Otherwise, one or more groups will find it impossible to access information in the document. Satisfying this checkpoint is a basic requirement for some groups to be able to use Web documents.
[Priority 2]
A Web content developer should satisfy this checkpoint. Otherwise, one or more groups will find it difficult to access information in the document. Satisfying this checkpoint will remove significant barriers to accessing Web documents.
[Priority 3]
A Web content developer may address this checkpoint. Otherwise, one or more groups will find it somewhat difficult to access information in the document. Satisfying this checkpoint will improve access to Web documents.
Some checkpoints specify a priority level that may change under certain (indicated) conditions.
|
|
This creates 2 problems:
1.Prioritizing conformance levels as a tester, developer, manager;
2. W3C WAI members developing success criteria don’t have uniform, standardized, instructions on how to determine appropriate conformance level of a new success criterion.
Having gone through the process during our 2.2 revision I felt like the conformance level determination process was very ad hoc. I also feel like the determination of what gets approved as a success criterion is very ad hoc.
|
|
This WCAG 2.x definition does not define anything about any of the conformance levels. It simply says A is minimum A must be satisfied in addition to AA to conform to AA and don for AAA:
Conformance Level: One of the following levels of conformance is met in full.
* For Level A conformance (the minimum level of conformance), the Web page<https://www.w3.org/WAI/WCAG21/Understanding/conformance#dfn-web-page>satisfies<https://www.w3.org/WAI/WCAG21/Understanding/conformance#dfn-satisfies-a-success-criterion> all the Level A Success Criteria, or a conforming alternate version<https://www.w3.org/WAI/WCAG21/Understanding/conformance#dfn-conforming-alternate-version> is provided.
* For Level AA conformance, the Web page satisfies all the Level A and Level AA Success Criteria, or a Level AA conforming alternate version is provided.
* For Level AAA conformance, the Web page satisfies all the Level A, Level AA and Level AAA Success Criteria, or a Level AAA conforming alternate version is provided.
*
<https://www.w3.org/WAI/WCAG21/Understanding/conformance#conformance-requirements>
Understanding Conformance | WAI | W3C<https://www.w3.org/WAI/WCAG21/Understanding/conformance#conformance-requirements>
w3.org<https://www.w3.org/WAI/WCAG21/Understanding/conformance#conformance-requirements>
[favicon.ico]<https://www.w3.org/WAI/WCAG21/Understanding/conformance#conformance-requirements>
|
|
From the understanding conformance document The success criteria were assigned to one of the three levels of conformance by the working group after taking into consideration a wide range of interacting issues. Some of the common factors evaluated when setting the level included:
|
|
Again, you have to define Level A equals this Level AA equals this. etc. all of these citations are just general
|
|
FWIW, I agree with @Helixopp that there should at least be a very broad definition for what constitutes a Level A, AA, AAA SC - along similar lines as the broad WCAG 1.0 priority definitions, as currently that's not properly spelled out. (also, hope you don't mind @Helixopp, but I edited your email follow-up comments to remove the previous messages that they were replying to) |
|
No worries. Thanks for the support. I would rather the definitions be detailed and specific, rather than general. Ideally that definition would explain publicly what requirements were met for any SC to be deemed a particular conformance level. That same definition would govern our process in nominating snd approving SC’s for any particular conformance level. Given that WCAG is a standard that process should be standardized.
|
|
There is a very clear definition of what goes in each level. If you are looking for a formula - or set of checkboxes -- you won't find them. There were many different factors that went into each decision. I can't speak definitively for 2.1 and 2.2 but for 2.0 -- in each case (almost every case) there were those that wanted to put a provision into another level than where it ended up. In the end - it was the level that everyone could reach a consensus on. That everyone could accept after sometime short and sometimes long debate and discussion(s). There was even debate about having two or three levels. At one point it looked like a majority wanted to go to 2 levels. But when asked it turned out half wanted to put level AA into level A and half wanted to put Level AA into level AAA. In the end -- what people could agree on was the three levels with the SC distributed as they were. I won't repeat all the different considerations -- since many of the are listed already. But those are just a subset of the 30 or 40 different things that people brought up in arguing for an SC to be in one level vs another. |
|
I understand that. But without a formula the process is arbitrary. What you described was an arbitrary negotiating process. Not a scientific or credible method that gives defining characteristics to each conformance level. That’s a huge problem. The conformance levels must be defendable. It’s not defense say well we all agreed on it.
|
Something like the above, combined with an explanation that - in naive terms - "Based on these considerations, Level A SCs are more critical, Level AA slightly less so, and Level AAA tend to not apply to all situations and impose limits..." or something. No doubt, that would be contentious when spelled out like that, though... Maybe at the very least having something like the above bullet points though that give an insight into the rationale for having different levels. |
|
Thanks, Patrick.
“Maybe at the very least having something like the above bullet points though that give an insight into the rationale for having different levels.” – This is what we need. Something that says Level A conformance is assigned to success criteria that....
Then list all aspects of that explanation that are uniformly applied to all Level A success criteria.
And do the same for the other conformance levels. But you can’t just say, well here are 5 or 6 considerations that we take into account for every conformance level, then we vote, negotiate, or whatever, on which level to assign each S.C.
That is arbitrary, inconsistent, and not standardized.
I do find this one ridiculous, but whatever: (that is, the knowledge and skill to meet the success criterion could be acquired in a week's training or less)
|
I'm going to suggest that the only time this is relevant is when the working group is coming up with new success criteria (and I've asked similar questions at that point). From an author perspective, one chooses the conformance level that meets the needs of one's client (or the regulations that are in effect), and one completes all the criteria that are indicated for that level.
Do you have examples of this? That is not my experience. I understand there was intention by some of the participants in 2.0 that level A criteria would be considered more critical to meet (and therefore have a higher level of severity, or at least priority, when not met); however, there is not any normative language that supports this. Further, in reality, the primary measure of severity tends to be a subjective understanding of its impact on users, either in regard to the degree it renders content unusable by specific users (such as those reliant on the keyboard API), or in regard to the number and types of users affected. Finally, the context of a specific page has a great deal to do with the relative severity cause by failing any one criterion's requirements. |
|
I would highly disagree that this is only important when developing new success criteria. For countries, municipalities, governments, etc. that require a specific conformance level as part of a law, this may be of less importance, but that’s not all countries, and not the United States.
Furthermore, laws can be changed. If someone where to bring to the attention of EU parliament, that WCAG conformance levels were arbitrarily established, without any uniformity, or criteria by level, it could easily be argued that the law needed change.
Aside from that, as professionals, for the W3C’s own credibility, this is important. We shouldn’t just shrug our shoulders because WCAG got away with it.
It is this kind of issue that has given rise to the need for a WCAG 3.0 and confusion around comprehending 2.0. We understand it because we live and breathe it. We need to make it understandable for everyone else also.
3:30 into this video https://youtu.be/jC_7NnRdYb0?si=9H7wIbSjIuC_e_jT is an example.
|
The whole idea that there's Level A, AA, AAA, and that they are additive (to meet A, you just do As; to meet AA, you have to pass A and AA; for AAA conformance, you must meet A, AA, AAA) does set up a hierarchy, showing that A is more "foundational" than AA so more important to meet, etc. |
|
Most global standards and regulations call out A and AA which indicates to me that both are needed to bring a sufficient level of accessibility to disabled people. At the same time, folks always want to prioritize issues - clearly some things such as flashing content are highest priority because of the impact - but other requirements may also be priority based on the factors previously discussed. However, the standard itself can't really prioritize one persons need above another - so the standard can't really get into that - but others can try to make determinations based on impact and biggest benefit to user needs. |
|
My point is clearly defining what conformance levels mean, not how people use them to prioritize issues. The fact that people use conformance levels to prioritize issues reinforces the importance of clearly articulating uniform parameters for each conformance level and consistently applying them
|
|
Also that statement about global standards is not true. There are no global standards. The UN has the CRPD Treaty which only applies to U.N. countries. Furthermore, it only requires countries to have disability rights policies. It does not require them to conform to any WCAG level or even to use WCAG in their policies. Each country does their own thing. The EU has a regional EU policy but none of these are “global”. I understand the desire to vehemently defend WCAG but this is indefensible, which is one reason why we need a WCAG 3.0. At the very least we need to resolve this issue.
|
but then even with that, it shows that AAA SCs are considered "less of a priority" / "less important", so again this is evidence that there is a hierarchy (whether intentional or not) |
|
@Helixopp I'm going to dodge all the commentary so far and try to answer your original statement and questions. Proposed draft response
Correct, there is nothing published in normative text in 2.x. The Understanding levels of Conformance subsection previously referenced provides a framework, but no clear delineation. Rationales that went into level assignment between A and AA are not included in the specification and can only be inferred.
No, level A criteria do not have to meet a certain level of severity. There is no clear relationship between a criterion's stated level and the severity of an issue resulting from that criterion not being met. This would be impossible to provide, given the broad range of scenarios in which any criterion may be considered, or the degree to which a criterion's requirements are not met. As well, there appear to be no regulatory frameworks that assign increased penalties to non-conformance based on the level of a failing success criterion. If such existed, that could potentially be one way in which conformance level could affect someone's assessment. But even in such a hypothetical, that would arguably affect the priority the issue was given, not its severity. Without the context of the page, it is not feasible to assign severity.
Just as there is no correlation at Level A, there is none at level AA. Given that level AAA is not part of a regulatory framework in any known jurisdiction, failures of AAA are less severe strictly from a compliance perspective. The relative impact on any user (one way of measuring severity) of an unmet success criterion is again going to be influenced by any number of factors that are independent of the level classification. |
Given pointed past comments elsewhere about various failings of the standard by pretty much everyone involved in this thread, I don't think anyone involved could be accused of vehemently defending it. I think everyone involved has a fairly mature attitude towards a useful but imperfect standard, so I suggest we set aside such language. |
|
Comments such as “it is clearly defined” and it I used in “global standards” (therefore it’s good though) are actually defending its current state.
The majority of comments have been implying that it’s good enough, leave it alone.
In my opinion those are defenses. I am entitled to have my own opinions. Nothing in any of my comments are aggressive, or adversarial. They are constructive. Let’s not try to imply that I act otherwise, please. My reputation does not deserve that implication.
|
|
Let me put it this way. I am saying we need to define conformance levels clearly. We need to create a definitive structure. One that explains the methodology to the public which we also use when creating new success criteria. I know we are working on 3.0 now and 2.x is kind of by the wayside but it still deserves attention.
The prioritization aspect is not the important part of. Y issue. The definition of conformance level is the heart of it.
|
|
These are things we need to pay attention to while developing 3.0. This entire conformance level just seems so arbitrary. If a form of conformance level is used in 3.0 it needs to be clearly defined and applied uniformly across the development of all guidelines. I still think we should provide some kind of documentation in 2.x clearly resolving this lack of definition.
|
|
For the record: my concern isn’t really with how regulators apply it but more so general practitioners that aren’t really well versed in the industry.
|
|
Whew! Lots of context here! Getting back up to speed after the weekend! I'll provide a brief recap for those who, like me, might need it. Ask:Clarify the parameters that were used to determine conformance levels. Concerns:
References:
Main Argument:
Consensus:
Copy Considerations:
My Updated Copy:The severity and prioritization of accessibility issues should not be determined solely by WCAG conformance levels, but rather by the criticality of the content and how real users perceive and operate the given site.
|
|
RE: Separating priority and severity Aren't content, perceivability, and operability crucial for accurately assessing both severity and priority? If not, it might be more effective to identify a common denominator rather than prescribing individual assessment methods for practitioners. (E.g. assign severity by taking these steps, but determine priority by following these steps) Or, we can just address the one most likely to be misunderstood - severity (as stated in the original ask). |
|
To be clear, since I filed the issue:
The ask is to publish an errata, or disclaimer, that conformance levels are not indicative of accessibility issue severity or priority.
Reason: the reason is because conformance levels were not assigned based on any specific parameters or formula. Instead they were just decided by consensus. They were never assigned as a matter of severity or priority.
The point of this issue isn’t to come up with a definition for conformance levels or an explanation of how they should be treated. It’s too late for that
I believe the Working Group has already agreed to the errata/disclaimer resolution. We are just formalizing the wording.
It would be inappropriate to list specific business names. Many participants in the Working Group already agree that this misconception has been observed throughout their careers.
Sent from my iPhone
[**Note**: @mbgower removed the thread of the prior comments, which was appended to this comment, because it was triggered as an email response.]
|
|
Thank you for clarifying your request @Helixopp. I understand and respect that associating a business name with an issue like this could be harmful, even if the issue is widespread. I hope you can also respect that addressing problems is easier when we can discuss specific examples. Since you're familiar with the businesses and initiated the original issue, could you share how you might write the disclaimer?
|
|
Please review the thread. Examples have been provided. Suggested wording has been provided. I have responded that I would like to horizontally review the suggested wording with the Accessible Platform Architectures Working Group before we commit to the text. It will take a few weeks before we can calendar this discussion on APA's agenda. It doesn't matter how many businesses apply conformance levels this way. It matters that the text of WCAG makes this possible. The W3C doesn't hold anyone accountable for how they use our publications. |
I know because I have authored several of the iterations myself.
I specifically requested your version of the disclaimer copy because I genuinely valued your unique context and perspective @Helixopp . However, based on your response, I'm assuming that any comments made after your request to send this copy to the APA team have not been and will not be considered. If my assumption is incorrect, could you clarify which version of the text we are planning to present to them?
I'm not entirely sure what you meant in this part of your message. To clarify my earlier "side note," I was requesting specific figures for use in our working group discussions. |
Do you mean that you believe there is general consensus in the comments of this issue? I think it would be more accurate to say that several suggestions have been offered by different people as ways to try to address various comments you have made. But I haven't actually seen suggested wording from you, which could really help bring this issue to some kind of completion. Please note that the WCAG 2 Task Force has a process of vetting PRs and responses, before sending those on to the Working Group. The Working Group has not addressed this in the past to best of my knowledge. If it has, and you mean there is already a resolution to address this, please point to minutes of that resolution.
One can provide real world examples of situations without 'naming names.' I'd welcome those. That said, if someone has done this in a public VPAT or ACR, then I think it is appropriate to cite those. I haven't seen anyone in this thread agree that a misconception about levels by a page owner has led to significant problems with severity and priority ranking, as you describe. That's why I keep asking for details and examples. Repeatedly responding that it happens does not actually make it any more true. |
priority or the order in which defect A is resolved relative to defect B is meaningless without another metric to hash it with something suchb as frequency or the number of people affected - it is no less or more subjective than the assignment of severity. Similarly, the notion that developers are somehow swayed by higher or lower measurements of severity are without evidence. In the vast majority of situations, severity is defined for functional/systems testing, and priority is determined by product owners in a defect triage meeting or sprint ritual. The proposed wording is an addendum rather than an erratum as it seeks to clarify rather than correct, but either way I don't believe it will prevent people from drawing comparisons. A preferred approach might be to include an addendum that:
FWIW, a normative update to WCAG 2.3 tabling the parameters used and describing how consensus was reached for each success criteria would likely assist practitioners explain the W3's thinking. IMHO, it's the pushback from various stakeholders that is driving the perceived need for decisions about definitions to justify the relative importance of a given defect. It is not too late. It is very easy, for example, for a manaager to say that there is no such thing as a S1 or P1 accessibility defect because of the disconnect between functional/systems testing definitions of severity and priority and levels of conformance. By providing an explanation in black and white, at least practitioners have something to point to when they're being railroaded by indifferent management. From where I sit, the early draft of the WCAG 3.0 conformance model is only going to make this mismatch between specification and the expectations of so-called business stakeholders worse - if the relatively straightforward WCAG 2.x conformance model is being translated into what is more familiar, then this all-new conformance model witll fail utterly. |
an explanation is always better than no explanation ... there's never going to be numerically verifiable threshholds or objective measurements of user impact largely because of the nature of the subject matter, but it wouldn't be too hard to slap together a table using factors like populations of affected users in terms of broadest, broader, and broad, for example, or is assistive technology required to work around a failure of this success criterion: yes, sometimes; never etc. Even an overlay of Schedule B in EN501.349 would provide practitioners with something in black and white to back-in a priority decision about a given failure. As it stands, the Easter-island statue silence in WCAG 2.x serves no-one. |
|
It is very straight forward Definition of a Level A (or AA or AAA ) provision
If you want to know how they decide which level
If you want to know what criteria they used
So don't look for a formula or critical factor. Oh I can say that there was never a weighing of importance of one disability over another (except in effort -- we spent much more effort working on provisions for groups like cognitive, language, and learning disabilities since they were the hardest to find solutions for - since mostly we solved accessibility using AT (there are very few provisions for people who are blind -- but they had AT (Screen readers) so they could make materials accessible themselves as long as the information was compatible with AT. But for cognitive, language, and learning disabilities there was essentially no AT so that group could not make thing accessible for themselves like blind people could -- but just using the AT compatibility provisions. There are therefore many more provisions for direct access (no AT) for people with cognitive, language, and learning disabilities than for any other disability including blindness. In fact - we tried to get everything we could think of that would qualify (be testable) into WCAG 2.0. There never was a time when we excluded or downleveled a provision because of disability. Best g |
|
There is no question as to the extraordinary efforts of those involved in formulating what has pioneered an entire discipline - this is not about saying the original working group did something fundamentally wrong. Nor is it to say that WCAG 2.0 is no longer entirely fit for purpose. But, in my experience, just restating that this is the consensus a group of people arrived at for a given success criterion two decades earlier when being questioned by an indifferent and likely ignorant stakeholder is just not very persuasive when prosecuting a case for why this or that carries greater or lesser weight than something else. Noone could have predicted the way that WCAG 2.0 has been desseminated, applied, paraphrased, interpreted, or challenged no more than the scribblers of various holy scriptures could be thousands of years ago. I am sure you can see the clumsy parallel ... So, while there may not be a 'formula or a critical factor', a working group note fleshing out the explanation in the latter part of your comment with further explanation of what levels of conformance are not equivalent to or should not be correlated with, or addressing the 'churches' which have evolved with the specification would be - in my view, at least - more helpful than what is published now about how success criteria are assigned to levels of conformance. > It is very straight forward |
|
The explanations are already documented. We took great care to do that and mention all the major factors that were involved. There is not any more to say. There were a myriad of factors. But the strongest ones are already listed. when people ask for more -- they ask for determinants -- which were the "criteria" for being A or AA -- as if there was a formula or weighted list. Or that everyone used the same criteria for their decision as to which level. They didn't. We had advocates for every disability and sub-disability, from industry (and they had different views and priorities and thoughts ) and academics and developers and government and evaluators and just volunteers. Together they took each provision -- discussed at great length - and decided which level it would go in. Then they debated back and forth until they reached consensus on a level for each one. So that is why I said that the only definition of exactly why each one ended at a particular level is -- that they were put at the level that the working group reached consensus that it should be put at. Lots of people with lots of views on lots of factors -- and probably few if any people that put the same weight on the same factors in making their decision. So there were no rules or defining factors or formulas for deciding. So the best that can be done is to describe the key factors that people said they used (which is done in WCAG/Understanding WCAG). And to say that the final decision was the decision made by the group. Hopefully that makes it clearer. best |
|
thank you for taking the time elaborating how it went down back then, but, for me and presumably others, what is in the understanding documents just doesn't cut the mustard anymore for practitioners at the coalface so I guess I'll leave it there ... |
|
"why is this SC in level FooBar?" it keep reminding me of: “When I use a word,’ Humpty Dumpty said in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’" and to be clear, we're not asking for a definitive formula here, but clearly, the parameters/axes that led the group to a particular decision have been laid out. things like (but not limited to) "how severe the problem is for certain user groups", "how many user groups are affected", "how easy/hard it is to achieve a better outcome/implement a solution", "are there workarounds that make the problem circumventable". so clearly the axes have been defined, just that nowhere it has stated, in a simple way, how those axes influenced a decision: when an SC tended to affect more users, was easier to implement, and had no workarounds, it generally ended up in Level A; on the other end of the spectrum, if an SC tended to not affect as many users, or was harder to implement, or there were workaround (albeit laborious/convoluted) that made them less of a showstopper, then an SC tended to be lumped more towards Level AA or AAA. this seems fairly non-controversial statement (though of course not elegant, but far from a "definitive formula" which nobody has actually been asking for), but somehow there's a strong reluctance to acknowledge this for some reason... |
|
I've read all the comments, but there are so many now that I'm not sure if I've missed something:
However, a better definition is not possible because the SC levels are already assigned and even if we were to come up with a new definition, the existing levels would no longer fit. But they can't be changed (so easily). Therefore: Why not live with the fact that there is a definition that may not be perfect, but is still comprehensible. @patrickhlauke: The 3 levels are not just Humpty Dumpty ... |
|
@JAWS-test because that definition you link to defines the considerations/axes, but then does not actually define what A, AA, AAA are. it explains the considerations, but then misses the last bit that says how those considerations then map (even at a very broad/handwavy way) to A, AA, AAA. and i'm still surprised that some people are seemingly trying to gaslight folks about it I mean, even just a simple statement at the end of that definition that says "consider the Levels to be rough groupings of similar SCs (similar based on these various axes/considerations). these levels don't necessarily imply importance or severity or any kind of hierarchy, on their own" that would still not be satisfying (and arguably still not quite true), but at least it would provide that one missing bit from the definition. |
love the humpty-dumpty ... I was going to say 'fundamentalism' hence the holy scriptture analogy, but thought that they may be considered inappropriate, and I don't intend to be offensive. |
agreed @JAWS-test that it is what it is and we have to live with it, but this does not preclude providing an addendum or additional information rather than formulae. My central concern is the various 'churches' that have filled the void in lieu of a satisfactory explanation thereby making a nonsense of conformance altogether .... there is nothing preventing an explanation of the original process or directly addressing the ways in which - in my view - these levels are being misused in these growing orthodoxies in a working group note or whatever ... |
|
@patrickhlauke and @electronicwoft: ok, thank you very much for the answers. I can understand that. I.e. the point is that A should defined as essential, no workaround possible, easy to fix, without restrictions on the design AAA would be defined as: not important, there could be workarounds, difficult to fix, may affect the design or content ... AA is somewhere in between. It won't be so easy to find a good formulation for this. But I understand that the description of the process for determining the levels is not identical to the definition of the levels. However, in terms of content, the definition will not provide much more than the explanation at https://www.w3.org/WAI/WCAG22/Understanding/conformance#levels |
|
I'd dispute calling AAA "not important" in particular. It's there because it makes a material difference to some people with disabilities. My very rough mental model, that's now two-and-a-half decades old (going back to formulating the levels for WCAG 1.0), is: Level A are things that are outright showstoppers - if you don't do them, saying you've created an accessible resource is in the region of a bare-faced lie. Not having a tall narrow set of steps for a wheelchair user to get to the bathroom might be an analogy. Level AA are the things that can be worked around, but at great effort - some people with disabilities will be unable to effectively use a resource that fails one of these. Removing a couple of low steps that some people could help most wheelchairs over, adding a rough curb-cut, not having a ramp at a 35º angle. Level AAA are the things that make the difference between "more or less accessible" and "equality of opportunity" - putting the ramp front and centre of the entrance, a light gradient, broad enough to go in company, no corner so tight that many wheelchairs can't actually use it. When it comes to agreeing exactly where a given requirement fits, given that in different cases it will manifest in different ways and at different levels of severity, means there is no magic formula. Think about describing images: Sometimes it's crucial, like the map explaining where a shop is or the menu icon. Sometimes it's a picture of the sponsor's CEO, who actually looks like a stock photo, or an actual stock photo - probably a "conventionally pretty" woman because that's what the media still does - added to a newspaper article for no obvious reason. |
|
When we were working on WCAG 2.0, there was a document titled Requirements for WCAG 2.0. I think criteria for conformance levels were brought up in discussions about that document but they were not included in it. As Greg Vanderheiden pointed out in his 3 June comment, the conformance levels assigned to new criteria were based on what we could reach consensus on. One undocumented rule that we used is that anything that affected visual design would not be at Level A, which is why the most basic contrast requirements (WCAG 2.0's SC 1.4.3 and WCAG 2.1's SC 1.4.11) are at Level AA. There is also Requirements for WCAG 3.0; based on a brief look, this document does not discuss criteria for assigning conformance levels to success criteria. |
|
I think the main point of the thread is (or should be) to encourage people to apply severity appropriately. I'd suggest a little change to the last suggestion I could find:
Sidenote: When we report issues the severity is completely separate from WCAG level. I don't know about other agencies, but we've employed people who have worked at other agencies and none were surprised by this approach or asked why we didn't use WCAG levels. The only time I've come across people asking about whether to use the WCAG level for prioritization is development teams who are new to accessibility. In a scenario where they will have to work through all the A/AA issues anyway, it isn't the end of the world if they do the level A ones first. However, we do encourage them to use the severity and t-shirt-size columns to determine priority (so easy-to-fix blockers are first etc). |
|
@alastc that'd be a good start, yes. then, if it truly is a case of "SCs have been grouped into levels based on similarity (on the various axes of consideration)" it'd be good to mention that too, to make it even clearer that it's not a hierarchy of importance, but just a very rough set of buckets (which still doesn't explain why this was done - if #allSCsMatter - what each bucket actually represents, but it'd be better than nothing/conjecture) |
|
perhaps relevant, as the missing piece for me at least has been "yes, we understand that SCs were placed in categories/levels because the WG decided they should go there, but surely the WG must have had a working understanding or definition of what each category/level actually represents in order to make that judgement call" ... this seems a good starting point https://lists.w3.org/Archives/Public/w3c-wai-gl/2002AprJun/0266.html |
|
Let me start by first asking -- why do people want to know? Usually it is:
The answer to the first (which are more/most important) is
The answer to the second (which should I do first) is
OK that now raises the question:\ That is a great question - that is somewhat lost to time. Here is the answer in short - followed by more detail.
OK that explains level AAA (sort of). So why are there two more levels (A and AA) instead of just one more (other than AAA)? That is the one that is most misunderstood I think.
** So that leaves -- Why were there 3 levels instead of 2?** The bottom line is that -- we could not get any consensus when we tried to have just two levels
At one point - there was a concerted push to have WCAG look more like other standards - where there were only two levels: required and recommended (equivalent to SHALLs and SHOULDs.) There was even a majority of the group that wanted to have two levels. The problem was that half of the group that wanted two levels wanted to create it by collapsing everything in AA into AAA, and the other half wanted to collapse AA into A. So in the end - the one thing that everyone could agree on was the three levels. And the items in the three levels were determined in the same (consensus) manner. Each item was placed in the location where the working group finally put them.-- often after long discussions on multiple meetings -- and always using feedback from the many public-releases-and-requests-for-comments that we received. All the lists of ideas for how to sort them from the past and the lists that have been made retrospectively to explain them are all good indications of the factors that went into deciding which went where -- but they are not absolute determinants -- and you will find things that seem to not quite fit or not fit that set of criteria. That is because they were factors that were considered - not criteria. If you are looking for the absolute accurate description of why each item ended up in the level it did -- here it is.
That is the only accurate and detailed answer.
Looking back is only so helpful. And mostly just for the similar task that we will have for WCAG 3.0 As to using levels for the two questions at the top (i.e.,. "Which are more important ?" and "Where should I start?") the answer is DON'T.
But NEVER take levels as levels of importance to people with disabilities. They were NEVER that. Hope this is helpful. Best Gregg |
|
[edit: actually, I give up, this is just going further in circles and there's no real point here anymore] |
this is precisely what is wrong with the current situation in my view. if severity is determined by criticality then it is tautological and superfluous. As I have said before, as a training aid for developers it is at best vague and at worst misleading because it is a value plucked out of the sky by testers who - in my experience - wouldn't know their ass from a hole in the ground when assessing user impact ... So let's abandon severity once and for all and look at priority. "the importance of the content as well as how easy the site is to operate and understand" is neither readily testable nor especially practicable given that it is likely only confirmed by extensive usability testing which is a nice to have most organisations place behind accessibility in their list of priorities. What's more, 'the importance of the content' is, if you break it down' just another way of saying 'the chances of this or that preventing a user from completing a given user task is 100%' which is basically saying it's a Sev1 issue. And, again, not something I'd leave up to the average tester. So let's put aside priority as defined above also and concentrate on why a tester needs to assign a value to a defect in the first place or at all. The reason is so that whomever is resolving said defects then has what is effectively a chronology to help structure their workload. As soon as levels of conformance - whatever these might mean or how they are defined or by what process they were conceived - is removed from the calculation of this chronology, then the ultimate organisational goal of achieving a given level of conformance for a defined scope of web content is jeopardised. tester X says that a single failure of 1.1.1 is a S4 and hey presto it isn't resolved before production and, given the awkward nature of the WCAG 2.x conformance model, may mean that conformance requirements 1, 2, and 3 are not satisfied and any claim of conformance for that scope or part of it is no longer valid. It simply doesn't matter what each level of conformance means or how a given success criterion was assign to it - as soon as levels of conformance are obliviated by arbitrary assignments of severity or priority, or importance, or untested perceptions about ease-of-use, then the specifcation becomes valueless. This is why there must be another factor or factors that can be assigned by testers objectively such as frequency or scope or some other categorisation that don't rely on a moistened finger in the air or experience or sixth sense or divine guidance to formulate a chronology based on levels of conformance. Any proposed addendum must make it clear that traditional conceptions of severity-as-user-impact or criticality as just another name for severity or any other similar utterley subjective or untraceable or untestable factor is inappropriate for setting a chronology. |
that's sad to hear, Patrick - there is a point and it's to make WCAG 2.x more usable for the people who are stuck with it. |
|
if all levels are important and, above all, if there is no difference between A and AA, then WCAG 5.2.1 makes no sense. There, a precise distinction is made between
If they are all equally important, WCAG could also offer that someone achieves conformance with
|
|
the key there is ''equally' ... some are more equal than others! No-one has said that all success criteria are EQUALLY important, only that they are important ... which is about as useful as saying that they all are success criteria. this is part of the problem with this thread - terms such as severity, priority, criticality, importance, etc. are undefined ... |
and others
There does not appear to be any clear definition of what parameters a success criteria must meet in order to qualify for any particular conformance level.
In order to be considered Conformance Level A does a success criteria have meet a certain level of severity, as in if not met it will severely adversely impact the user.
Likewise is an AA Conformance Level less severe/impactful? And so on.
Or is it that Level A Conformance is attributed to easy fixes, slightly more complicated issues are deemed AA Level, etc.?
Accessibility consulting companies are assigning severity levels to WCAG Success Criteria (critical, moderate, serious, etc.)which seem to be largely based on conformance level.
This is extremely subjective, and dangerously misleading. Lack of transparency of what criteria are required to be met for each conformance level is partly to blame.
If such information does exist it is too difficult to find.
The text was updated successfully, but these errors were encountered: