Sound disappears sometimes in Full-Screen Mode

[rea-laura]

Test Device

Dirac

Operating System

El Capitan 10.11.6

Browser

Safari 11.1.2

Problem Description

For phetsims/qa#219:
Sound will sometimes disappear in full-screen mode. Once lost, the sound cannot be recovered by exiting full-screen mode, nor by pressing the sound on/off button or interacting with the sim in any other way. The entire webpage must be refreshed to fix it.

Issue only occurs on Safari browsers, and is hard to reproduce.

Steps to Reproduce

I have been struggling to find a way to reliably reproduce this issue. For the most part it seems like it happens when many items in the PhET menu are used consecutively, followed by full-screen mode. Sometimes though, it just doesn't happen at all.

Here are the two most reliable methods I've found for reproducing it (although they are not that reliable):

Method 1:

1. open sim.
2. press reset button, ensuring that sound works.
3. enter full-screen mode.
4. press reset button - if no sound, you've replicated the issue!
5. if sound still exists - exit full-screen mode.
6. repeat steps 3 and 4. If this fails to recreate the issue three times in a row, refresh the page and start over (from step 1).

Method 2:

1. open sim.
2. move all three sliders.
3. press reset button.
4. use the PhET menu to take a screenshot.
5. open screenshot.
6. close screenshot and go back to the sim.
7. without touching any other buttons, open the PhET menu and go to full-screen mode.
8. test for sound.
9. if this fails, refresh the page and try again.

[JRomero0613]

Can confirm that the second method for reproducing works on Mac 10.13.6 as well, have not been able to reproduce using the first method though.

[jbphet]

@rea-laura and @JRomero0613 - I mentioned this to someone yesterday, and they said that they'd run into a similar problem when displaying vids on youtube. Have you checked to see if you can cause this problem on non-PhET pages? I'd like to make sure it isn't a general Safari bug before investing time in it.

[KatieWoe]

Confirming that this occurs on 10.14 as well. Played around a bit with youtube but haven't seen it there yet. When this occurs it does not stop voice over from reading.

[emily-phet]

I googled around looking to see if others were posting about Safari/sound/full screen issues. I didn't find anything.

@jbphet if you decide to try to fix this, try to limit the amount of time spent searching for the issue. This sim needs to move forward and it looks like both of the ways to reproduce this issue involves use of the PhET menu - which is a very small use case.

[jbphet]

I am able to duplicate this on Dirac (the same machine on which this was initially discovered) using "Method 2" described above, but only for built versions, i.e. it does not happen in RequireJS mode.
I added some debug code and was able to see that the state of the audio context is changing from running to interrupted at some point, and once the audio context is in the interrupted state, sound won't play anymore. This is very odd, because according to the Web Audio spec, the only valid values for the state are running, suspended, and closed, so it would seem that this version of Safari is violating the Web Audio specification.

Here is a simpler procedure to duplicate:

Method 3:

1. Make sure Finder is running
2. Load the sim in Safari
3. Press the reset button (it should produce sound)
4. Switch the active app to Finder
5. Go back to the sim in Safari and go straight to Full Screen mode without any other interaction
6. Press the reset button in Full Screen mode, it likely won't play sound
7. Exit Full Screen mode and press the reset button again, and it still won't produce sound

[jbphet]

From what I can tell, the audio context seems to be going into the interrupted state shortly after the sim goes into full screen mode. No additional interaction with the sim is necessary.

[jbphet]

Fixed this in phetsims/tambo@2825e91, but I mentioned the wrong issue in the commit.

[jbphet]

There is a fix in place for this, but if my understanding of the problem is correct, it's actually a bug in Safari and the fix is a workaround. I am going to leave this issue open to remind myself to log an issue with Apple.

[emily-phet]

Sounds good - note that @jessegreenberg has a place for documenting a11y-related bug reports and known issues. Please check with him about where to record this info.

[jbphet]

Unassigning while we await the results of the RC test.

[KatieWoe]

Have not seen in 1.6.0-rc.2. @jbphet it sounded like you still wanted this open, so reassigning to you.

[jbphet]

Nope, no need to keep it open if testing shows that it's fixed. Closing.