Elements can become "un-interactable" on iPad

[DevonQui]

Test device
iPad Air 4th Gen

Operating System
15.2

Browser
Safari

Problem description
This is for phetsims/qa#771

I noticed that when you create a new tab (leaving the simulation) while still interacting with an element in the play area, that element will stop being interact-able once you return to the play screen. I believe this is a multi-touch issue but I'll try to see if I can reproduce on my laptop as well and rename the issue if needed.

Steps to reproduce

1. Click on any of the sections
2. Drag an element over the play area
3. While finger is still on element, click new tab '+' button on top right
4. Go back to play area and notice how you can't interact with element anymore

Visuals

Troubleshooting information:
!!!!! DO NOT EDIT !!!!!
Name: ‪Number Play‬
URL: https://phet-dev.colorado.edu/html/number-play/1.0.0-dev.45/phet/number-play_all_phet.html?screens=1,2,3
Version: 1.0.0-dev.45 2022-01-31 11:53:19 UTC
Features missing: touch
Flags: pixelRatioScaling
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15
Language: en-US
Window: 1512x865
Pixel Ratio: 2/1
WebGL: WebGL 1.0
GLSL: WebGL GLSL ES 1.0 (1.0)
Vendor: WebKit (WebKit WebGL)
Vertex: attribs: 16 varying: 30 uniform: 1024
Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 32)
Max viewport: 16384x16384
OES_texture_float: true
Dependencies JSON: {}

[DevonQui]

I noticed that this affects multiple simulations and I tested on master and the published sims on the website.

[chrisklus]

Thanks @DevonQui! I think this issue should be moved to scenery if it affects multiple sims. I'll assign @jonathanolson to see if this is a known problem and/or if it should be worked on.

[jonathanolson]

Not known to me. I believe iOS used to send an "interrupt" when this happened, so that the touch should be stopped. Confirmed this on my iPhone.

[jonathanolson]

Also tagging @kathy-phet to see if this should be a higher priority. It seems like it could make things feel broken often, and I'd like to see if I can find a simple fix.

[kathy-phet]

Yes, I think you should look for a solution @jonathanolson. Thanks!

[jonathanolson]

@KatieWoe can you test previous versions of iOS? I'm curious whether this is a regression. iOS used to send pointercancel events properly (and were the best platform for making sure pointercancel events happened), and now in this case we are NOT getting them (or the lostpointercapture).

iOS (15.3, Safaru 604.1)

Win Chrome (properly working :

Win Firefox (uses touch events instead of pointer events):

[jonathanolson]

If we have any older iOS (or touch-based Safari) devices to test this on, it would be very valuable to see if this is reproducible.

I'll also be checking some older simulations.

[jonathanolson]

Pointer events are currently disabled on Firefox due to #837. Disabling pointer events on Safari seems to work around this issue (but may cause others).

[jonathanolson]

Index: js/input/BrowserEvents.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/input/BrowserEvents.js b/js/input/BrowserEvents.js
--- a/js/input/BrowserEvents.js	(revision 3cd5832001ab0cbdb29808f0036a120a489483f5)
+++ b/js/input/BrowserEvents.js	(date 1644294036714)
@@ -123,8 +123,9 @@
    * @private
    *
    * NOTE: Pointer events are currently disabled for Firefox due to https://github.com/phetsims/scenery/issues/837.
+   * NOTE: Pointer events are currently disabled for Safari due to https://github.com/phetsims/scenery/issues/837.
    */
-  canUsePointerEvents: !!( ( window.navigator && window.navigator.pointerEnabled ) || window.PointerEvent ) && !platform.firefox,
+  canUsePointerEvents: !!( ( window.navigator && window.navigator.pointerEnabled ) || window.PointerEvent ) && !platform.firefox && !platform.safari,
 
   /**
    * {boolean} - Whether pointer events in the format specified by the MS specification are allowed.

[jonathanolson]

@KatieWoe can you also let me know if the patched version available at https://bayes.colorado.edu/dev/olsonjb/number-play_en_phet.html (with the above patch) seems to show the issue? It seems fixed on my phone.

[jonathanolson]

I'm curious in particular if there are regressions for how the touch events work there (e.g. dragging a finger outside of the sim bounds). I'd probably need to create a more comprehensive test (including iframes) if initial testing looks good.

[KatieWoe]

I cannot reproduce the bug in the link above on iPadOS 15.3