VoiceOver Screen Reader Issues with Slider Read-Outs

[JRomero0613]

Test device: MacBook Pro

Operating System: macOS High Sierra v. 10.13.4

Browser: Safari 11.1

Problem description: For phetsims/qa#125 There are a couple of issues associated with the VoiceOver screen reader on macOS on Safari. The first bug makes it so that the screen reader will read out the value of the slider, but not the resistance. The second bug makes is so that the screen reader will read out one slider value and resistance, but after any other adjustments to the slider only the resistance is read out. The third bug makes it so that only first slider value is read out and any adjustments to the slider afterwards produce no value for the slider or resistance.

Steps to reproduce:
For the first bug:

1. Load the sim by opening link in a new tab
2. Navigate to a slider using the tab key
3. Adjust the slider using any method
4. The screen reader will read out the value of the slider, but not the resistance

For the second bug:

1. Load the sim by clicking link (do not open in a new tab)
2. Navigate to a slider using the tab key
3. Hit control-option-shift-up arrow
4. Adjust the slider using any method
5. The screen reader will read out the first slider value and the resistance as intended
6. Adjust the slider again using any method
7. The screen reader will not read out any subsequent slider values, only the resistance

For the third bug:

1. Load the sim by opening link in a new tab
2. Navigate to a slider using the tab key
3. Hit control-option-shift-up arrow
4. Adjust the slider using any method
5. The screen reader will read out the slider value only
6. Adjust the slider again using any method
7. The screen reader will not read out any subsequent slider value or resistance values

Screenshots:
For the first bug:

For the second bug:
My screen recorder did not allow me to open the sim by clicking the link so the recording starts in the sim, note that you must navigate to the sim via the link and not by opening the link in a new tab.
Google drive link: https://drive.google.com/open?id=1LuZUnMQc2z2ifB99PhV_STxq3WVNlaY9
For the third bug:
Google drive link: https://drive.google.com/open?id=11LEG2BeJ3Fi_IEfA4zeOzg5dhs0pl_9E

Troubleshooting Information:

Name: ‪Resistance in a Wire‬ URL: https://phet-dev.colorado.edu/html/resistance-in-a-wire/1.5.0-rc.5/phet/resistance-in-a-wire_all_phet.html Version: 1.5.0-rc.5 2018-05-31 01:31:21 UTC Features missing: touch Flags: pixelRatioScaling User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15 Language: en-US Window: 1680x989 Pixel Ratio: 2/1 WebGL: WebGL 1.0 (2.1 INTEL-10.32.48) GLSL: WebGL GLSL ES 1.0 (1.20) Vendor: WebKit (WebKit WebGL) Vertex: attribs: 16 varying: 15 uniform: 1024 Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 16) Max viewport: 8192x8192 OES_texture_float: true Dependencies JSON: {"assert":{"sha":"928741cf","branch":"HEAD"},"axon":{"sha":"cc053b4d","branch":"HEAD"},"brand":{"sha":"89d28f63","branch":"HEAD"},"chipper":{"sha":"54f17be2","branch":"HEAD"},"dot":{"sha":"0dd6ee8e","branch":"HEAD"},"joist":{"sha":"f047fb1b","branch":"HEAD"},"kite":{"sha":"b6071478","branch":"HEAD"},"phet-core":{"sha":"f35ff65e","branch":"HEAD"},"phet-io":{"sha":"d54be499","branch":"HEAD"},"phet-io-website":{"sha":"28284828","branch":"HEAD"},"phet-io-wrapper-classroom-activity":{"sha":"c84e3046","branch":"HEAD"},"phet-io-wrapper-lab-book":{"sha":"ebf7c7dc","branch":"HEAD"},"phet-io-wrappers":{"sha":"ce57c3e2","branch":"HEAD"},"phetcommon":{"sha":"a5a7478c","branch":"HEAD"},"query-string-machine":{"sha":"485e174e","branch":"HEAD"},"resistance-in-a-wire":{"sha":"f80c6036","branch":"HEAD"},"scenery":

JG EDITS: For readability

[jessegreenberg]

Thanks @JRomero0613, also assigning @terracoda to review.

Bugs 1 and 3 look very similar to me, though in bug 3 aria-valuetext is also broken. Basically aria-live isn't working after opening the sim in a new tab. This is probably very related to the VoiceOver bug we found where switching tabs causes aria-live to break: phetsims/a11y-research#91

The second is new, but also feels like VO bug.

I think the next step will be to try to reproduce these in vanilla HTML outside of a sim. @zepumph would you mind creating these tests? Do you have access to VoiceOver at all? If not feel free to reassign to me.

[terracoda]

@jessegreenberg, I was just typing that I will test this. Thanks for the assignment.

[terracoda]

My Testing Results:
MacOs High Sierra 10.13.4
Safari 11.1

First Bug

• I am able to reproduce when I start Voice Over before opening the sim in a new Tab.
• the alerts work just fine if I start Voice Over after the sim has loaded in a new Tab.
• I agree with @jessegreenberg that this is the VO bug described in aria-live alerts break in VoiceOver when switching tabs and leaving iframe a11y-research#91

[terracoda]

My Testing Results:
MacOs High Sierra 10.13.4
Safari 11.1

Second Bug

• In the steps to reproduce, step 3 says, "Hit control-option-shift-up arrow". Why would I do that? That command is supposed to take me out of the web content.
• When I do step 3, Voice Over tells me that I have gone out of web content, however, my keyboard focus is still on the slider. I stop getting slider values, but still get alerts when I move the slider. Presumably I get alerts with no slider values because I am no longer, "in the slider", even though I can still make it move.
• When I press control-option-shift-down arrow after doing step 3 and then using the slider, my visual focus disappears, and my virtual cursor moves to the content of the H1 and I can now read with the virtual cursor. Pressing Tab key after I am back "in web content", my focus moves to the slider and I get values and alerts again as I should.

My Thoughts
I don't think users would actually do step 3. But since they can, I think this is a focus management bug, if it is a bug. The focus of the slider is visually maintained and operationally maintained after the user has used a VO command to exit the slider. It seems like the virtual cursor has gone to the live region where the alert content is because when I use command-option-shift-down arrow to go back into web content, my cursor focus shifts to the h1 and visual focus disappears.

@jessegreenberg, is it possible to listen for this VO command and remove focus from the slider, perhaps shifting the virtual cursor to the group heading, Slider Controls?

[terracoda]

My Testing Results:
MacOs High Sierra 10.13.4
Safari 11.1

Third Bug

• I can reproduce the silence.

My Thoughts

• The lack of alerts is due to the known VOice Over bug (opening in new tab).
• The lack of slider values is due to leaving web content with the Voice Over Command, command-option-shift-up arrow.
• Re-entering the web content and restarting Voice Over gets things back to normal.

If we can better manage focus when VO's command-option-shift-up arrow is used, we should be good.

[terracoda]

@zepumph, I am unassigning myself. Do you have enough to go on?
Is it possible to place the virtual in the PDOM on the group's heading and remove visual and functional focus from the slider when a user presses command-option-shift-up arrow?

Please re-assign if you have questions.

[jessegreenberg]

is it possible to listen for this VO command and remove focus from the slider, perhaps shifting the virtual cursor to the group heading, Slider Controls?

@terracoda wouldn't that change prevent default behavior for command-option-shift-up arrow, and make it impossible for the user to leave the web content that way? I am not sure if it is possible to prevent default behavior of the virtual cursor.

Right now, we aren't doing anything special with focus management in this case. My best guess right now is that VO fails to "blur" the document and element with focus when leaving web content, and so the arrow keys continue to manipulate the slider, thought VO has stopped observing things like aria-valuetext and aria-live, so we dont' hear those updates.

[terracoda]

@jessegreenberg, I certainly do not want to create a trap. I think what I was trying to suggest, is that we force VO to "blur".

I do not actually know what "blur" is. What I see and hear is that keyboard focus remains when it likely shouldn't.

If this means it is Voice Over bug, then that's ok with me, but I was wondering if it was safe or possible to force a "blur" in this case.

[zepumph]

I agree with @jessegreenberg that preventing default behavior of that key command seems dangerous, but also is it even possible? Will VO pass those keydown events to the browser? Or will they just eat them up because they are first on the software stack?

[terracoda]

@jessegreenberg and @zepumph, and in case I wasn't clear. I don't think that users would intentionally do step 3 upon getting started with the sim.

I'll just read a little bit about focus/blur before posting more thoughts.

[terracoda]

@jessegreenberg and @zepumph, and I should also point out that when I do step 3, I do get a beep from VO, which I think is an error beep. This beep is likely useful in telling users that something is not quite as it should be.

[terracoda]

Here's an article from WebAim, you might already be aware, Accessible Javascript

Quoting from the article:

Typically, these events [onFocus and onBlur] do not cause accessibility issues unless they are modifying the default behavior of the web browser or are interfering with keyboard navigation within the web page. Examples of such issues might be [1] automatically setting focus to other page areas with onFocus and onBlur, [2] trapping the user inside a form control, [3] dynamically revealing form controls immediately upon a user leaving (blurring) a form control, etc.

It seems like we are not doing any of the things listed above, which is good. Visually and operationally, however, focus is remaining on the slider when a VO user intentionally leaves the slider with command-option-shift-up arrow.

@jessegreenberg and @zepumph, how do you determine what is causing the problem?

1. Is Voice Over failing to tell Safari that a blur event has occurred (VO BUG)? Or,
2. Is Safari not recognizing Voice Over's command, command-option-shift-up arrow (i.e., Safari BUG)? Or,
3. Both 1 and 2? Or,
4. Is it something we are doing when we detect focus in the PDOM and draw the pink focus highlight on the corresponding visual sim element?

From the user's side, I am not sure what is happening.

EDIT: I changed the phrases to questions ;-)

[terracoda]

@jessegreenberg, @zepumph I posted some questions to the WAI interest group list and I got 4 replies that might shed some light on this problem, and help us understand if we need to change anything.

[terracoda]

Core content of my initial email to WAI:

In recent Quality Assurance testing of a prototype for the Resistance in Wire simulation, we found an issue when using Voice Over. It seems that Voice Over does not successfully blur (i.e., remove focus) from a slider control that has focus when the user uses the native Voice Over command, command-option-shift-up arrow, to exit web content.

The result is that the slider remains visually focused and operational via all the normal keyboard methods, even though the user is technically “out of web content”. In this case, Voice Over will not announce slider values if the user adjusts the slider. In this case, Voice Over only announces aria-live alerts. This means that the user is not getting all the information they are supposed to be getting when they adjust a slider. In addition, if the user re-enters web content using command-option-shift-down arrow, their virtual focus starts at the top of the sim with the H1 content.

We think the lack of blurring is a Voice Over bug.

My questions to the group are:

1. Are we supposed to do anything special to make Voice Over blur focus in the case, or is this indeed a Voice Over bug?
2. Where is virtual focus supposed to go after a properly blurred event?
3. Is starting at the top of the page the correct place for the virtual cursor to be after a successful blur event, after successfully exiting web content with the native Voice Over command?
4. Starting at the top, seems to be the default behavior. Is it the correct behavior?

We have done a lot of user testing with blind users. Exiting web content after focusing a control is not something that has occurred during usability testing with Voice Over users.

It would be great to get some thoughts from experts on this list regarding this blurring issue.

Response # 1

I am not as familiar with Voiceover on Mac OS, but it is likely similar to iOS and how screen readers work on Windows. When using the screen reader’s virtual or review cursor, it doesn’t trigger focus or blur events. The idea of the virtual/review cursor in most screen readers is to allow the user to navigate the current contents without triggering changes to the user interface.

It is typically system focus that triggers blur and focus events. A screen reader typically only manipulates the system focus when it is not in virtual/review cursor mode but in some type of form or application mode.

That is likely the problem you are dealing with now.

Response # 2

I agree with [above response], exiting from web content navigation I would not expect a blur event because system focus would remain on that control unless the VO cursor focused another control.

Response # 3

The command you mention is the stop interacting command. This will move the VoiceOver location to a containing object which will depend on if Group or Dom navigation is enabled on exactly what the VO cursor will then be looking at. Unless a containing div can also have focus (perhaps via tabindex) focus will not move when doing this command because the focusable will still be in the VO cursor though perhaps additional focusable objects might be contained in the VO cursor.
Focus change will occur when the VO cursor interacts with another focusable item, or a tab/shift tab is typed.

Though if QuickNav is on with quick keys commander enabled, then when stop interacting is run, quickkeys should be re-enabled allowing navigation similar to Virtual Cursor in JAWS.

More to details to come

[terracoda]

My follow-up to # 1

Thanks for your comments. These are helpful.

In the case, I described, a Voice Over user is Tabbing to an interactive control, a native range input (i.e., a slider). On focus, the slider does have “system focus”, it has a visual indication of focus and is operational. Adjusting the slider at this point, the user will get the proper experience (slider values and aria-live alerts). If the user exits web content with Voice Over’s command-option-shift-up arrow, system focus is not blurred.

Is it the responsibility of the web application to blur system focus, in this case, or is it the responsibility of the screen reader software and browser?

As far as I know, we are not preventing Voice Over from blurring.

Response # 1, follow-up reply

Based on what you described, I would not expect the application to handle the situation where a screen reader user decides to switch to virtual/review mode—especially since the main purpose of that mode is to allow the screen reader user to navigate through the user interface without making any changes to it.

More details to come.

[jessegreenberg]

Thanks @terracoda - It sounds like responses are indicating that focus is supposed to stay on the slider when leaving the web content, is that how you interpret it too?

[terracoda]

@jessegreenberg, yes, but there's a bit more to the story ...

[terracoda]

My follow-up to # 2

System focus does indeed stay on the slider.
A user has to re-enter web content navigation for the slider to work properly.

Can I assume, that "system focus" and "keyboard focus” are the same thing? I find talking about screen reader modes and focus quite tricky, and I have read a lot on the subject.

Response # 2, follow-up

That’s generally correct. Screen readers can take over the keyboard such as with quick navigation keys but VO should try to synchronize it’s cursor with the system/keyboard cursor in most situations. But in other situations the VO cursor and keyboard/system cursor are not synched.

[terracoda]

My follow-p to # 3

And thanks to [responder # 3].
Yes, I am talking about the stop interacting command.

I think you have nailed the problem here.

Focus disappears when the virtual cursor moves to an un-focusable element after the stop interacting command, and focus reappears when the virtual cursor moves onto another focusable element.

We do have containing elements, there are three sliders in a list.
We may need to group the sliders more explicitly for Voice Over to really stop interacting.

This is all awesome information that I can share with my developer colleagues.
Thanks everyone.

No response yet.

[terracoda]

Last follow-up to all

Thanks [to all] for your help.

[Responder # 1], with regards to your last comment, where would you expect your virtual cursor to be, once you entered virtual/review mode?

I think [Responder # 3], mentioned it should be near by, like at the containing element.

In our case, review mode seems to be starting at the top of the document at the H1, which seems odd to me.

No response yet.

[terracoda]

@jessegreenberg and @zepumph, I think I copied everything I have received into the issue.

Based on Response # 3 in #153 (comment), I am wondering if the natural "Group or Dom navigation" is being adversely effected in some way, possibly from the tabindex= -1 on everything?

[terracoda]

I think we have implemented everything correctly.

I am just a little unsure on why the virtual cursor goes to the H1 after exiting web navigation on a slider, which is quite a ways down the page?

Where does the virtual cursor go, if you exit forms mode while a slider has focus using NVDA?

[terracoda]

@jessegreenberg and @zepumph, I did a couple of other tests with the simple testing environment, and system focus remains on the slider and the slider remains operable, even when reading with the virtual cursor. However, changes to values on the slider in this in-between state are not read out after I do the "stop interacting command". What is read out is where the virtual cursor is, which makes sense.

Of course, in the simple test environment, there are no live regions, so no alerts fire wen I move the slider while I am in "non-interactive mode".

The only questions that remain in my mind are:

1. Should we (or can we stop the alerts from firing) in this in-between state if the user uses the Arrows keys without their screen reader keys (VO keys). Basically, can we silence the slider all together (values and alerts) even if its value changes?

• Is there any way for us to detect if the user has chosen to be in the "non-interactive mode"?

2. Where is the expected re-entry spot for the virtual cursor when a user intentionally stops interacting with a focusable interactive element?

• Should it be nearby, or is it ok that the user re-enters the web content at the H1?
• Note this is only a problem when the user has focus on the slider, but has not formally entered the slider with the Voice Over command, command-option-shift-down arrow.

3. Is there anything we are doing to prevent Voice Over from re-entering nearby the element where the user stopped interacting. In particular, is the tabindex=-1 having a side effect?

Personally, I think it would be great to have answers to these questions. I do think, however, this issue is not a show-stopper for RIAW. We are managing system focus/keyboard focus correctly as far as I can tell.

I will reach out to the WAI one more time.
@emily-phet, do you think we should spend any more developer time on this issue. I have spent a lot of time on it already.

[terracoda]

@jessegreenberg, and @zepumph I did a little more testing with the simple Slider test environment with tabindex="-1" on all the non-interactive elements. I found that:

• tabindex="-1"` does not have an effect on where Voice Over places the virtual cursor upon re-rentry into web content in the test environment.
• Re-entry almost always starts on the parent un-ordered list element as long as system focus has not moved off any one of the sliders, after interacting stopped.
• Re-entry almost always starts at the H1 if system focus has been lost from the page.
• Re-entry can start at a non-interactive element, like the paragraph of help text if system focus was there when the user stopped interacting and has not been lost from the page.

In the sim environment, re-entry always starts at the H1. It may not be typical behaviour, but it is consistent. As mentioned above, I will send one more email to the WAI, to see if it is desirable to silence live region alerts after a user stops interacting, and if there are any known techniques for doing so.

[terracoda]

For the record, see a11y-research repo foe the simple slider test page.

[terracoda]

New questions, Response # 1

What might be useful for debugging purposes is to try the VoiceOver command control-option-F4 this should tell you what has focus. So if you stop interacting then you could try the keystroke and see where VoiceOver believes focus to be.

Also if you get to that point there is an accessibility-dev list hosted by Apple that has active Apple programmers on it.

Adding

Actually, in Macintosh VoiceOver the VoiceOver cursor will not move to the JS changed focus in several circumstances.

1. When “group mode” is enabled.
2. When command-option-shift-F3 is typed to temporarily unlink “tracking”
3. When VoiceOver and keyboard focus tracking is disabled in the VoiceOver utility.

I could go on in length about group mode web navigation, but I don’t think it is widely used.

This tip may also be relevant to phetsims/balloons-and-static-electricity#401 (comment)

[terracoda]

This issue is not just for sliders. It's for anything. You always re-enter at the H1, when the VO stop interacting command takes you "out of web content".

[terracoda]

New questions, Response # 2
Note the nice bug categories.

I want to make sure I understand something. When you talk about re-entering the web content, do you mean that you either switch to another application than the browser or another browser window and then back or that you close the browser window and then relaunch the browser/site again?

If either of those is the case, I do not think it is anything you need to solve. When in virtual/review mode in a screen reader and you leave an application and return, where the virtual/review mode return to on the page will depend wholly on whether or not the screen reader remembers where it was on that page. There are probably a number of rules internal to a screen reader regarding where the review mode focus starts. Actually, the only one to be concerned with would be if someone is using a screen reader to view a page and you use javascript to move keyboard/system focus to an element that can receive keyboard/system focus, the review mode cursor should move to that spot. Elements that can receive focus are basically all the HTML form elements like input, button, links, select, etc. The only other elements that can receive focus are those with tabindex=”0” or tabindex=”-1”—where -1 means that focus will only move to that element if you use javascript to move focus there.

I am less certain about the aria-live issue. Where I work, we frequently do accessibility testing with iOS but not with the Mac OS. I know we have observed issues with how aria-live regions work in Safari on iOS—situations where the live region is read but it doesn’t look like the aria-live region should have updated.

If the aria-live regions are speaking when Voiceover is not directly interacting with the page (i.e., it is in review mode,) you might want to see if something is triggering the code that generates the messages for the aria-live region. If nothing appears to be causing that code to generate a new message, you might be looking at what we call a user agent defect/behavior.

We (my team) categorizes issues as user agent defects/behaviours if the browser, screen reader, or operating system is causing the problem rather than something in the application causing the problem. We call it a user agent defect if the the issue seems to break our standards. We call it a user agent behavior if it is something we expect the browser, OS, or screen reader to do, but one of our testers thought it was a defect.

At that point, we typically have a discussion about the impact of the user agent defect on the users and the, then we decide whether or not we will try to fix the problem. However, at the moment, I don’t know if the aria-live issue you are dealing with now is a user agent defect or not.
...
level of effort to work around the issue (or if it is even possible)

[terracoda]

After all the testing and discussion, I think we can defer this issue until it becomes an issue Voice Over users.
Reasons

• I think it is an unlikely scenario that a user will keep interacting with the slider after they have intentionally decided not to.
• If they do, the sim does indeed change, and the alerts are giving them the correct alert information.
• We are managing system focus correctly in this scenario.
• It is easy to recover, they just get their focus back in synch with Tab or by actually reviewing content until they hit an interactive element.
• The web content re-entry point is consistent across all interactions.

I recommend that we do not fix until we have evidence this is a problem for users.
@emily-phet, do you agree?

[terracoda]

Just adding last comments
More confirmation that we are good to go:
My thank you to WAI list members

[Responder #2], by ”re-entering the web content", I just mean toggling in and out of virtual/review mode with Command-Option-Shift Up Arrow and Command-Option-Shift Down Arrow. In our sim environment, the user always re-enters at the H1, when the stop interacting command (Command-Option-Shift Up Arrow) takes them “out of web content, Resistance in a Wire”. It doesn’t matter where they were when they stopped.

Regarding the live region alerts, they fire only if the slider is moved/adjusted. They are not randomly being fired. It makes technical sense as I am actually moving the slider, but since I told Voice Over that I wanted to stop interacting with the slider, Voice Over is not reading out slider values.

I think we have implemented it all accessibly, and we just need to assess whether or not it is feasible, possible or even needed to make a work around to silence the live region alerts.
[...]

Last response from Responder # 2

I Agree.

While I dislike the approach Voiceover seems to be using for live regions, I think it is justifiable not to try to fix that problem since the user is the one that initiates the change in mode which causes the live region not to read. In that case, it is also likely to be something that users are aware of—and it always helps if the user is aware of the behavior.

With regards to the other situation of where Voiceover re-enters the web page, I agree. It just seems to be a behavior of Voiceover. Personally, I would only be concerned about it if you were using code that failed to conform with some spec and that was causing the issue.

I did get to try out your simulation. I think it looks pretty good.
[...]
Taliesin, thanks for sharing your UI with us. I mostly work on insurance and banking related user interfaces so it makes a nice change of pace to look at UI’s from other domains.

[emily-phet]

I agree - no more action needed here.

[terracoda]

Adding last comment from Responder # 1 that came in after closing issue:

My understanding is that keyboard focus is on the slider and that when VoiceOver and Keyboard cursors are on the slider it speaks the new value but when you adjust the slider with the keyboard and VoiceOver is not on the slider it does not provide feedback. So to me the question is should you be getting feedback on a user adjusted slider when the VoiceOver cursor is indirectly or not pointing at the slider. I think it should still be providing feedback.

My response (no answer yet)

[Responder # 1], that is indeed the problem and question.
And to be clear, the user is getting some feedback, just not all the appropriate feedback. They get live region alerts with slider values. When VoiceOver is not on the slider it silences readouts of the slider values, but does not silence live region alerts.

Do you know of a way to force the value readouts? I would not want to rely even further on live region alerts, though that seems like a possible solution on the surface of it.

Would you consider this a VoiceOver bug, or an application bug? We do not silence the value readouts, VoiceOver does.