Boundary / Verbose examples

[Jym77]

Playing around the idea of boundary examples as discussed in #2164.

This adds one "boundary example" to some of the rules, showing how the rules are partial check and stuff that is inapplicable or even passing might still fail the SC (or other requirements).

These "boundary examples" must not be part of the normal examples that are required for tools to claim "consistent implementation" of the rule. Thus, tools can be more clever than the rule (i.e. fail these cases), but still have a consistent implementation.

This PR only shows a possible way to show these examples: in a collapsible section after the normal examples (so they are still together with the rule itself, but do not take space for people not interested in them). If we decide to go that way, we would need to build the website code (and tweak the validation) to automate part of this process.

I do believe that this won't require a format update since the format doesn't prohibit "extra" examples. Another possibility would be to place the boundary examples in the Background, but that makes them show a bit earlier in the text of the rule (even if the section is collapsed).

Closes issue(s):

• N/A

Need for Call for Review:
This will require a 2 weeks Call for Review (new concept)

---

Pull Request Etiquette

When creating PR:

• Make sure you're requesting to pull a branch (right side) to the develop branch (left side).
• Make sure you do not remove the "How to Review and Approve" section in your pull request description

After creating PR:

• Add yourself (and co-authors) as "Assignees" for PR.
• Add label to indicate if it's a Rule, Definition or Chore.
• Link the PR to any issue it solves. This will be done automatically by referencing the issue at the top of this comment in the indicated place.
• Optionally request feedback from anyone in particular by assigning them as "Reviewers".

When merging a PR:

• Close any issue that the PR resolves. This will happen automatically upon merging if the PR was correctly linked to the issue, e.g. by referencing the issue at the top of this comment.

How to Review And Approve

• Go to the “Files changed” tab
• Here you will have the option to leave comments on different lines.
• Once the review is completed, find the “Review changes” button in the top right, select “Approve” (if you are really confident in the rule) or "Request changes" and click “Submit review”.
• Make sure to also review the proposed Call for Review period. In case of disagreement, the longer period wins.

[dan-tripp-siteimprove]

I have some thoughts:

1:
The name "boundary examples" might not cover everything. "Verbose examples" might. It's true that "boundary examples" covers most of what we'll want to do with these. But not everything. All of your concrete examples in this PR fit into the category of "boundary examples". However I can picture another kind of example which is not "boundary", and which might still be worth putting in this new section in the future. Those would be examples in which there is no rule / SC discrepancy, and which might be already covered by previous examples in the main examples section, and which are still valuable to include because they are non-obvious. "Verbose" would be a decent word for these. Like these examples from my long 'label in name' comment:

• <a href="#" aria-label="Some article by John Doe"><h6>Some article</h6><p>by John Doe</p></a>
• <div role="button" aria-label="Hello world"><p>Hello</p><p>world</p></div>
• <button aria-label="how are you"><span>you</span><span>how</span><span>are</span></button>

Someone might say that they're redundant, and they would have a point. But they weren't always redundant - some (all?) of them used to fail. So such examples will help future archeologists, as well as some implementors and users operating in the present.

Another set of "arguably redundant" examples which are not "boundary examples" and would have value as "verbose examples":

• <a aria-label="WAVE" href="#">W A V E</a>
• <a aria-label="Call 1 2 3. 4 5 6. 7 8 9 0." href="tel:1234567890">123.456.7890</a>
• <a href="#2021" aria-label="20 21">2021</a>

2:
The new headings might cause confusion. It would be unfortunate if someone confused the "Passed Example 1" under "boundary examples" with the "Passed Example 1" under the main "passed examples" section. It seems to me that headings should be unique on a page when feasible.

---

That said, in general I like this a lot. In particular, the ideas of 1) putting it in a collapsible section, and 2) putting that section after the current examples.

Let me know if I can help implement the website code for this.

[giacomo-petri]

Suggested change

These examples are not normative and not require for consistent implementations. They illustrate limitations and boundaries of the rule. Passed and Inapplicable examples in this category are likely to fail the rule's accessibility requirement, showing how the rule is only a partial test. Testing tools and methodologies are encouraged to be more clever than the rule, and to report the correct outcome for these examples. Therefore, they are not required for consistent implementation.

These examples are not normative and not required for consistent implementations. They illustrate limitations and boundaries of the rule. Passed and Inapplicable examples in this category are likely to fail the rule's accessibility requirement, showing how the rule is only a partial test. Testing tools and methodologies are encouraged to be more clever than the rule, and to report the correct outcome for these examples. Therefore, they are not required for consistent implementation.

typo, applies to all examples

[giacomo-petri]

I believe there is no need to clarify the outcome of each specific example for the specific rule (passed, failed, inapplicable) within a heading.

By definition, boundary examples "are not normative and are not required for consistent implementations." Therefore, in my opinion, and as recommended by @dan-tripp-siteimprove, even though I understand the intention to emphasize whether the current scenario is passing or inapplicable, we should avoid distinguishing this in the headings hierarchy to prevent confusion.

The reason why an example is an edge case, in my opinion, should be provided in the description of the example, leaving just "Boundary Example 1," "Boundary Example 2," etc., as titles.

[giacomo-petri]

And sorry, I forgot to mention the most important thing: I really love this idea!
We should definitely move forward with this idea!

[daniel-montalvo]

Suggested change

	<sumary>## Boundary Examples</sumary>
	<summary>## Boundary Examples</summary>

[daniel-montalvo]

Suggested change

	<sumary>## Boundary Examples</sumary>
	<summary>## Boundary Examples</summary>

[WilcoFiers]

Just leaving my 2 cents.

1. I don't think we should put details / summary in here. I'd rather have that be added by our build tool. It's less error prone that way, and means we can more easily make changes to it in the future.
2. I think we should call these "Miscellaneous examples". As I understand the conversation, we'll want to use this for a variety of different kinds of examples. The only thing that they have in common is that they're not passed / failed / inapplicable examples.
3. I feel we should use different H3s / H4s here. Calling these a passed example, or an inapplicable example doesn't set it apart from the rest of the rule sufficiently in my opinion. People will get confused by what these passed examples mean compared to the other passed examples. We should call them something else. There are lots of things we can use here, like "Out of scope example 1", "Accessibility support example 2", "Explicit semantic role example 3", "Shadow DOM example 4". etc.

[Jym77]

Just leaving my 2 cents.

1. I don't think we should put details / summary in here. I'd rather have that be added by our build tool. It's less error prone that way, and means we can more easily make changes to it in the future.
2. I think we should call these "Miscellaneous examples". As I understand the conversation, we'll want to use this for a variety of different kinds of examples. The only thing that they have in common is that they're not passed / failed / inapplicable examples.
3. I feel we should use different H3s / H4s here. Calling these a passed example, or an inapplicable example doesn't set it apart from the rest of the rule sufficiently in my opinion. People will get confused by what these passed examples mean compared to the other passed examples. We should call them something else. There are lots of things we can use here, like "Out of scope example 1", "Accessibility support example 2", "Explicit semantic role example 3", "Shadow DOM example 4". etc.

1. Definitely. This was just to show how it can look like.
2. 👍
3. I think it can be good for tools vendors to have a way to retrieve these cases and use them for validation purpose. This will require us to provide the rule's result in a way that the build tool can understand and forward. But I agree that the names can be confusing.