[COGNITION]: Links must be announced to screen readers as links. Currently being announced as "messages" or just text

[ianpsant]

Please view our Technical Support Guide before filing a new issue.

Screenshots

Version

<meta name="botframework-directlinespeech:version" content="4.12.0">
<meta name="botframework-webchat:bundle:variant" content="full-es5">
<meta name="botframework-webchat:bundle:version" content="4.12.0">
<meta name="botframework-webchat:core:version" content="4.12.0">
<meta name="botframework-webchat:ui:version" content="4.12.0">

Describe the bug

As a screen reader user, I need links to be announced as links, not "message" or read out as plain text.

Steps to reproduce

1. JAWS and VoiceOver are reading out links in the VA chatbot responses as "LINK TEXT, message" or just reading out the link text and never saying "link". I was able to trace this to the four layers of aria-roledescription attributes that have been applied to various chatbot containers.

Expected behavior

1. Remove all of the aria-roledescription attributes
2. Add an aria-roledescription="link" attribute to the link markup (not preferred)
3. Links announce themselves as "LINK TEXT, link" correctly in all major screen readers

Additional context

We do not have control over the aria-rolesecriptions that are applied to the parent elements of each link.
Corresponds to this VA GitHub issue: department-of-veterans-affairs/va.gov-team#25160

[Bug]

[compulim]

Technical investigations:

• Unrelated to aria-roledescription
• <ScreenReaderActivity> is turning links into plain text
  • When narrated, "Read more in this article."
  • This is done through activityAltText.js/htmlTextAlternatives() function
• EXPECT: When narrated, "Read more in, link, this article." (Windows/macOS) or "Read more in this article, link." (iPadOS)
• When computing alt text, today:
  • Today: This is a cat. <img alt="Funny cat" /> -> This is a cat. Funny cat.
  • EXPECT: `This is a cat. Image, funny cat." or "This is a cat. Funny cat image."
• Different browsers concat the accessible name differently
  • Windows Narrator or macOS VoiceOver: "Read more in, link, this article."
  • iPadOS VoiceOver: "Read more in this article, link."
  • Should we adapt? How?
  • How about internationalization? Comma may not be a universal way in I18n.
  • How many accessible names should we support?

Thoughts:

• We should modify activityAltText.htmlTextAlternatives to compute the text alt better
  • We did not compute "Accessible Name", e.g. "link", "image", or other aria-roledescription

[compulim]

I created a sandbox and tested this in multiple browsers. It seems a browser issue.

https://compulim.github.io/experiment-aria-active-descendant/

We are using ARIA active descendant to control how messages are focused.

The sandbox mimics the DOM tree of the transcript we use in Web Chat, notably:

• The transcript container is <div aria-activedescendant="IDREF" role="group" tabindex="0">
• All messages are in <li> under a <ul>
• When a message is selected, a random ID will be assigned to <li> and referenced by <div aria-activedescendant>

When using various screen reader to read the content of the page, we found that:

• ❌ Edge (Windows Narrator) + Chrome (NVDA) did not narrate the link text at all, seems like a bug
• ❌ Firefox (NVDA) do narrate the link text, but not link role (i.e. accessible name)
• ✔️ macOS Safari do narrate the link text and link role, but it requires extra gesture
• ✔️ iPadOS Safari do narrate link text and link role
• ✔️ Android do narrate link text and link role

[compulim]

Continue some investigations... it seems the root cause is use about aria-labelledby. Created another sandbox to simplify the investigation, https://github.com/compulim/experiment-accessible-transcript.

For example:

<div aria-labelledby="id-1">
  <div id="id-1">
    Here is <a href="https://wikipedia.org/">an article</a> to Wikipepdia.
  </div>
</div>

If the screen reader is reading the label for this element, such as, through aria-activedescendant. The screen reader will flatten the label as "Here is an article to Wikipedia.", instead of "Here is, link, an article to Wikipedia."

The "link" accessible name is not narrated in this case.

[compulim]

Trying to rationalize how screen reader read rich content.

<button type="button>
  Click <a href="https://bing.com/">here</a> to read the article.
</button>

After press TAB to focus on the button, the screen reader will narrate the whole content as plain text: "Click here to read the article." Instead of "Click, link, here to read the article."

Tested on both Edge (Windows Narrator) and Safari (VoiceOver).

I am biased to say this bug is by-design of screen reader. But I am yet to find some WAI-ARIA articles talk about how to compute text alternatives, to see if accessible name (a.k.a. aria-role) should be included in aria-label (a.k.a. text alternatives).

It seems screen reader only care about accessible name if the user scan/browse the content using CAPS + arrow keys.

[compulim]

Things we tried

We tried to remove aria-roledescription and it did not work for us.

We tried to add aria-roledescription for links, it did not work either.

Both did not work because Windows Narrator and NVDA flattened the message and it ignored aria-roledescription.

How Web Chat works with screen reader

There are 3 modes Web Chat will narrate the content of the message:

• As message arrive (live region)
• Focus on message
• Narrate the whole transcript

As message arrive (live region)

The plain text version of the message object will be narrated. Using plain text will avoid bugs in browser causing repeated narrations with screen reader. Please see https://github.com/microsoft/BotFramework-WebChat/blob/main/docs/ACCESSIBILITY.md#ux-live-region-transcript.

Focus on message

After focusing on the message in the transcript using UP/DOWN arrow keys, screen reader will narrate the message object as flattened plain text. Links inside the message will not be narrated as "link" as they are flattened by the WAI-ARIA Accessible Name Computation spec.

iOS VoiceOver never flatten messages because end-user cannot use arrow keys to focus on message. This mode does not applies to iOS VoiceOver.

Narrate the whole transcript

When using screen reader to read the whole transcript, screen reader will narrate all the details of the message object, including the tree structure and widgets inside the message object.

Workarounds

Using aria-details

We looked at aria-details attribute which does not flatten the message object. However, when we test it in Windows Narrator and NVDA, both are ignoring the aria-details attribute. (It does not read the content referenced by the element of the aria-details attribute.)

Adding hints to plain text message

Today, for message that contains interactive element, such as links, buttons, or input fields. Screen reader will narrate "Click to interact".

We may want to update this hint by indicating links, such as "Links in this message. Click to interact."

Known issues

Today, when screen reader narrate the whole transcript, it narrate each activity twice. This is being tracked in #4022.

Every message object will be narrated as plain text first, followed by message object details.

@ianpsant Is it possible that this behavior is the bug you are reporting?

I am happy to continue the discussion.

I built a sandbox for experimenting with screen reader behavior on transcript. Please visit https://compulim.github.io/experiment-accessible-transcript/ to try it out. If you think there are anything we could try out, please let us know.