Multitouch listener canceling/interaction system

[jonathanolson]

We'll need to support canceling of drags and button presses when another action is triggered.

Example 1: You are dragging something and hit "reset all" with another finger. The drag should be canceled. Also applies to 'down' states for buttons, etc.

Example 2: (From ABS phetsims/acid-base-solutions#117) We want the ability to go down on a button, but then have a touchSnag (a) allow to be started even though the pointer is active, and (b) cancel the button press state.

Example 3: You are pressing down on a button with one finger, but then press down another finger and attempt to pinch-zoom. We want the choice of whether to cancel the button, or prevent the pinch-zoom behavior.

Example 4: Moving something in the play area, but another finger switches the screen (what you were dragging is now invisible), and we want to cancel the drag.

Example 5: Using a slider (say on a faucet node) to control the rate of flow, but when X is full we want to cancel the slider manipulation and set the slider to a rate of 0.

For implementation, when a pointer is active (controlling something), we'll want to keep a reference (somehow) to it on the pointer. Other listeners need to be able to inspect what the pointer is doing to make decisions. Then we'll want the ability to interrupt() the pointer and have its listener cancel (adding in better "cancel" behavior to SimpleDragHandler and ButtonListener). We'll also need an easy way to iterate through pointers and determine which ones need to be interrupted (say, for the "Reset All" button).

[jonathanolson]

Also phetsims/joist#70

[samreid]

I'm dealing with circumstances like the above in Bending Light, by forwarding events to another listener, I'm having to hack the event.currentTarget like so:

            // HACK ALERT.  Changes the event's currentTarget.  Why didn't we need to do this with the other nodes?
            // Presumably because they were in similar coordinate frames?
            // See Scenery #131 create drag listener
            // See Scenery #218 multitouch
            // There is a precedent for this hack in SimpleDragHandler.js
            var c = event.currentTarget;
            event.currentTarget = prismNode;
            prismNode.movableDragHandler.forwardStartEvent( event );
            event.currentTarget = c;

[jessegreenberg]

phetsims/charges-and-fields#20 deals with this as well. That sim specifically runs into Example 1 above while dragging the meters:

Example 1: You are dragging something and hit "reset all" with another finger. The drag should be canceled. Also applies to 'down' states for buttons, etc.

[pixelzoom]

Running into this issue again with function-builder, need to cancel in-progress interactions when "Reset All" is pressed. Labeling for "developer-meeting" to discuss.

Skype conversation:

[1/11/16, 1:05:15 PM] Chris Malley: Anyone know the preferred method of canceling a SimpleDragHandler drag cycle if (for example) the "Reset All" button is pressed?
[1/11/16, 1:16:05 PM] Jonathan Olson: simpleDragHandler.endDrag( sceneryEvent ) should do it
[1/11/16, 1:16:21 PM] Jonathan Olson: preferably pass the scenery event that caused they reset all button click?
[1/11/16, 1:16:45 PM] Jonathan Olson: also that assumes it is being dragged actively
[1/11/16, 1:44:20 PM] Chris Malley: What does "being dragged actively" mean?
[1/11/16, 1:44:52 PM] Chris Malley: Does that mean you shouldn't call handler.endDrag if it's not in a drag sequence?
[1/11/16, 1:45:16 PM] Jonathan Olson: correct, I believe
[1/11/16, 1:45:31 PM] Chris Malley: Can I just test SimpleDragHandler.isDragging?
[1/11/16, 1:46:08 PM] Jonathan Olson: yes
[1/11/16, 1:46:28 PM] Chris Malley: I cannot get the Reset All button's event without creating significant spaghetti. Alternative?
[1/11/16, 1:46:56 PM] Jonathan Olson: try null?
[1/11/16, 1:47:08 PM] Chris Malley: I have something that is being dragged, and I want to end the drag if its dispose is called.
[1/11/16, 1:49:32 PM] Jonathan Olson: that's something SimpleDragHandler wasn't designed for, and sounds like a job for me (#218)
[1/11/16, 1:50:44 PM] Chris Malley: yes, This is a general problem, ubiquitous wrt the Reset All button.
[1/11/16, 1:51:08 PM] Chris Malley: I hate to be handling it on a case-by-case basis if there's going to be a general solution.
[1/11/16, 1:51:22 PM] Jonathan Olson: there should be a general solution
[1/11/16, 1:51:36 PM] Chris Malley: agreed. but that issue is from April 2014.
[1/11/16, 1:51:56 PM] Jonathan Olson: and if it's raised in priority, I can work on tackling it

[pixelzoom]

Two things to discuss at developer meeting:

(1) Should this feature be elevated in priority, since it needs to be handled in all sims? When will it be available? Who should work on it? Etc.

(2) Should I ignore this general issue in function-builder (as we seem to have done for most/all sims), or do I need to handle it in a sim-specific manner until this feature is available?

[samreid]

Jan 14 meeting: @jonathanolson will look into it

[pixelzoom]

1/14/2016 dev meeting:
AP: spend some time on it
JO: will create a proposal
CM: will ignore this issue for function-builder, in anticipation of a general solution

[jbphet]

The Isotopes and Atomic Mass (IAAM) sim recently underwent some dev testing, and several issues were logged that relate directly to having no general way to cancel multitouch listeners. Most of these IAAM issues have to do with a reset or a mode change while simultaneously dragging something, such as an atom. The issues are:

• Sim dies after placing atoms on nature's mix screen isotopes-and-atomic-mass#34
• Invisible neutrons can contribute to isotope mass isotopes-and-atomic-mass#36
• Reset button problems - Atoms isotopes-and-atomic-mass#42
• Reset button problems - Sliders isotopes-and-atomic-mass#44

What's the status of this issue, and how close are we to having a proposal to address it?

[jonathanolson]

Was looking into approaches for automatically "interrupting" any active listeners beneath a Node tree (e.g. entire sim screen). For most listeners (button/drag), seems appropriate to have an array of listeners for each pointer that would need to be interrupted.

Also need to determine if it's acceptable to act like it's an "end" of a drag, or if we need a callback for canceling (which we would also move 'cancel' events to trigger).

[pixelzoom]

2/11/16 developer meeting: Deferred by @ariel-phet, not a common use case. Not a concern for IAAM, may revisit for Function Builder.

[samreid]

I removed the deferred label because #131 is full-speed ahead.

[jbphet]

I noticed that I was added to the list of assignees, and I think it is to respond to what @pixelzoom said above, specifically:

It would also be good to get input from other devs on what patterns (if any) they've been using as a workaround.

I looked through the code where @pixelzoom is using endDrag( null ) (I found it in Function Builder and Unit Rates), and I haven't been doing anything like this. For the most part, I've been identifying and logging multi-touch cancellation issues, but not doing much to address them.

I'm unassigning myself, but I know that we discussed integrating the updated drag listener into Expression Exchange as a test bed once it is ready, so @pixelzoom or @jonathanolson - please reassign if that was what you were after.

[pixelzoom]

@jbphet You were added to the list of assignees because of this decision in yesterday's dev meeting:

• AP: Should someone take new multitouch for a test drive?
CM,SR,JB will review scenery#218 and scenery#131 sometime in the next couple of weeks. Will not integrate into Unit Rates. Might integrate into Expression Exchange.

I don't see that you've reviewed this in the context of #218, assigning you again.

[pixelzoom]

Highly unlikely that I'll be able to review this before the first week of April. If @samreid or @jbphet want to review before then, go for it.

[pixelzoom]

@jonathanolson I see a bunch of commits and assignment to me by you, but no notes about what was done, or what you'd like me to review. Can you give me some direction?

[jonathanolson]

Review of DragListener/FireListener/PressListener and the example demo would be helpful. Available over Skype to discuss together or to give an introduction if that would help.

[pixelzoom]

@jonathanolson how is that different from what you wanted in #131?

[pixelzoom]

Is "the example demo" scenery/examples/input.html, or something else?

[jonathanolson]

Sorry, did the work for this at the same time as #131, forgot which issue I was in.

Is "the example demo" scenery/examples/input.html, or something else?

Yes, that is the main demo. It handles canceling of inputs with the provided button (or switching of the in-example tabs).

For interruption, this is handled in Node.interruptInput/interruptSubtreeInput, and Pointer (multiple attachment methods).

[pixelzoom]

Commits and comments for this issue appear to be combined (unintentionally?) with #131. So I'm going to give all of my feedback in that issue. Removing my assignment.

[samreid]

What is the relationship between this issue and #131? I'll plan on giving my feedback in that issue unless I hear otherwise.

[jonathanolson]

What is the relationship between this issue and #131? I'll plan on giving my feedback in that issue unless I hear otherwise.

They were implemented at the same time, but #131 more deals with the actual listeners side of things, and this issue deals with the Pointer/Node/Input support for the interruption.

[jonathanolson]

Note to self: Look into the interruption of Display listeners. When should this be done? Is it needed?

[jonathanolson]

The above commit finished the last remaining item to be done above. Closing.