Locale doesn't switch back to English-speaking voice with secondLocale=es on safari

[Nancy-Salpepi]

Test device
MacBook Air (m1 chip)

Operating System
12.0.1

Browser
Safari

Problem description
For phetsims/qa#771 and related to #31:
When changing the locale back to English, the voice doesn't change back to English. Instead the Spanish-speaking voice says the English words pronounced like Spanish.

This seems to be a Safari issue. It did not occur with Mac + Chrome or Mac + Firefox.

Steps to reproduce

1. add the query parameter secondLocale=es
2. Go to the Ten screen and press the megaphone button to hear the English-speaking voice say "zero"
3. Toggle to "Espanol" and press the megaphone button again to hear the Spanish-speaking voice
4. Toggle back to English and press the megaphone button again--it is still the Spanish-speaking voice
5. Change the amount of objects to one.
6. Press the megaphone button. "One" is pronounce Oh-nee.

Visuals

secondLocale2.mov

secondLocale3.mov

Troubleshooting information: !!!!! DO NOT EDIT !!!!! Name: ‪Number Play‬ URL: https://phet-dev.colorado.edu/html/number-play/1.0.0-dev.45/phet/number-play_all_phet.html?screens=1,2,3&secondLocale=es Version: 1.0.0-dev.45 2022-01-31 11:53:19 UTC Features missing: touch Flags: pixelRatioScaling User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.1 Safari/605.1.15 Language: en-US Window: 1409x687 Pixel Ratio: 2/1 WebGL: WebGL 1.0 GLSL: WebGL GLSL ES 1.0 (1.0) Vendor: WebKit (WebKit WebGL) Vertex: attribs: 16 varying: 30 uniform: 1024 Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 32) Max viewport: 16384x16384 OES_texture_float: true Dependencies JSON: {}

[chrisklus]

Thanks @Nancy-Salpepi! @jessegreenberg and I discovered that this issue was caused by my workaround for having preference for Chrome voices. We fixed that behavior over in phetsims/scenery#1282, and removing the workaround in Number Play seems to have fixed the problem. Ready for confirmation on phettest.

[Nancy-Salpepi]

The sim would freeze periodically when I tried to press the speech synthesis button.

[jessegreenberg]

@Nancy-Salpepi did you notice this error in master or in 1.0.0-dev.45?

[Nancy-Salpepi]

@jessegreenberg I was on Master using a built version so that I could use that query parameter.
I was using the all_phet_debug link

[jessegreenberg]

Great, thanks! This assertion is in my court, assigning to me.

[jessegreenberg]

I believe #134 (comment) has been fixed, @Nancy-Salpepi can you please confirm?

[Nancy-Salpepi]

Hey Jesse
After pulling all, building a new version, clearing my history and using a private browser I still see this issue on Mac + safari. Button noises (like reset all) still occur, but nothing actually happens in the sim. I can't move any objects. When the freeze occurs seems to change.

numberplayfreeze.mov

[jessegreenberg]

I am hitting this again while working on phetsims/utterance-queue#65

[jessegreenberg]

I think this was fixed in phetsims/utterance-queue@4be055c.

I pulled everything and tested in Safari on a Mac with the all build with ?secondLocale=es and so far I have not been able to produce this problem. I also tried wiht the ?countAloud param.

I AM seeing this on iOS Safari, but I am not convinced that it isn't testing cached code.

EDIT: OK I was able to clear the cache and I see this happening on iOS Safari. So there is a form of this issue still around.

[jessegreenberg]

I am failing to get some end events on iOS Safari. Perhapas we need to reopen phetsims/utterance-queue#52

EDIT: Actually, I am failing to get the 'start' event too, we call synth.speak() but nothing happens. synth.speaking is false and we are ready for the next utterance. But we didn't dispose of the previous utterance because we didn't get any events.

[jessegreenberg]

So far I have tried

• Removing the "engine wake" workaround we added for ChromeBooks.
• Adding back the "safari memory" workaround we removed in Can we remove safariWorkaroundUtteranceWrappers from voicingManager? utterance-queue#52.

It seems to break most consistently on the first usage of SpeechSynthesis with speakIgnoringEnabled. If it works there then it breaks after cancel.

[jessegreenberg]

OK I finally think I got to the bottom of this in phetsims/utterance-queue#66. I tested on macOS Safari and iOS Safari and saw no crashing in number-play. @Nancy-Salpepi can you please make sure this error is no longer happening?

[Nancy-Salpepi]

@jessegreenberg things look good when I stick to one screen, but if I am in the first screen in Spanish and then switch to the second screen that is in English--the voice is still Spanish. Toggling back and forth fixes the situation.
Also, if the voice is English and then I switch to another screen that is set to Spanish--it is the English voice speaking in Spanish (with a really bad accent--especially on the compare screen).

NPlocale2.mov

[jessegreenberg]

Alright! I think that means the crashing bug reported in #134 (comment) is no more but the original problem reported in this issue is not fixed. Will take a look.

[jessegreenberg]

This part of the bug happens on all browsers because there is one isPrimaryLocaleProperty per screen, but only one voiceProperty for the entire numberPlaySpeechSynthesisAnnouncer. When the screens change we don't update the voiceProperty of the Announcer.

It seems like this could be fixed by either updating the voiceProperty when the screens change. OR the isPrimaryLocaleProperty could be for the whole simulation and persist across screens.

@chrisklus do you have a preference?

[chrisklus]

Thanks @Nancy-Salpepi and @jessegreenberg! A teacher recently experienced the problem described in #134 (comment), and i've started working on it (my bad for not creating a separate issue yet). My current approach has been the first option that @jessegreenberg described:

It seems like this could be fixed by either updating the voiceProperty when the screens change

And I think that will give the desired behavior.

Thanks yall for all the work on this issue - i'll leave assigned to me and close when I make a new issue for the separate voice bug.

[chrisklus]

I opened #152 to address #134 (comment), closing.