Mobile VO starts in wrong place when entering screen from home screen

[KatieWoe]

For phetsims/qa#628
Seen with iPadOS 14 with VO. When entering second screen from the home screen, the focus seems to be on the tick mark options. When entering the first screen, it seems to be in the middle of the scene description.

Image.from.iOS.5.MOV

[zepumph]

I don't think this should block RaP publication, but it is good to have an issue noting it.

[terracoda]

@KatieWoe, had you already interacted with that screen, or was this the first time you entered the screen and you were entering from the Home page.

Ideally, focus should be placed on the H1 when the screen loads for the first time. I am not sure if that is still the case if a learner is returning to a screen.

[KatieWoe]

It has been a long time since working on this issue, but I would suspect it occurred the first time it was entered/there was no previous interaction, otherwise I think I would have mentioned it.

[zepumph]

I am not sure if that is still the case if a learner is returning to a screen.

This behavior occurs every time there is a screen change. I'm not sure why VO is acting up but I believe this was reported in another sim. @jessegreenberg does this ring any bells?

[jessegreenberg]

I encountered something like this recently in phetsims/greenhouse-effect#182 (comment) that case, focus should move to the observation window heading in greenhouse-effect after "start sunlight" is pressed. In that issue for iOS VoiceOver sometimes it worked and sometimes it didn't. When it didn't work, focus moved to the address bar.

For this case, I have a crazy theory that if "Choose range for tick marks" control is positioned under the screen button in the PDOM VoiceOver might try to put the cursor on it. Its just a hunch and I don't have any good ideas on how to address now if that is the case.

Sorry, I don't have more thoughts. Let me know if this is something we should spend time on.

[zepumph]

@emily-phet, should think block publication of RAP?

[emily-phet]

No, I don't think this should block publication.
It sounds like the following is the case:

• The issue is not specific to Rap
• The issue happens only some of the time
• When the issue happens, it's relatively clear a problem has happened (e.g., reading the address bar for GHE, reading from the middle of the scene summary)

If there was a category for: if we had time to at least pin down the source of the issue and/or fix while still publishing the sim before the end Q3 (so the end of September) I would put it in that category, but I'm guessing we don't have way of estimating how much investigation or a fix will cost us timewise, and I'd prefer not to find out if there's any risk that investigating this could push publication of RaP + Friction past Q3.

[terracoda]

I also do not think it should block publication.

Based on the structure of the PDOM, in theory (and likely in practice) an miss-placement of focus like this should be relatively easy to recover from....

• oh my focus is now here, that's weird. I want to be on X element.
• learner uses their screen reader to find X element and moves on.

Not a perfect experience, but one that a learner should be able to remain in control of.

[zepumph]

phetsims/greenhouse-effect#182 (comment) is closed, perhaps this is ready for testing. I'll take a look.

[zepumph]

I was not able to consistently reproduce this today. Sometimes it was at the top of the sim, and sometimes it went to the address bar. I never had it go to the control panel components. Recognizing #374 (comment) and our current timeline, I'm going to mark this as deferred.

[terracoda]

Is this still an issue with the latest release of MacOS?

[zepumph]

@KatieWoe can you see if you can reproduce it on master please?

[KatieWoe]

I checked on master with the latest iPadOS (now iPadOS 16) and did not see this. Let me know if you want further testing.

[zepumph]

That is excellent news! Thanks for checking. Closing