touch-action and scrolling directional lock

[graouts]

I’m looking into an issue reported against Safari in iOS 13 related to Pointer Events. I've created a test case that reduces the issue to a simple usage scenario where a page is designed to only ever scroll vertically but contains an interactive area where the user may scroll horizontally to go through various panels (think an image gallery). The interactive container can be scrolled by listening to pointer events and applying a CSS transform to reflect the pan gesture. That interactive container also has touch-action: pan-y.

In Safari on iOS 13, since the page scrolls vertically, the interactive area will get a pointercancel event during a horizontal panning gesture unless the user is careful not to stray from a very straight horizontal panning gesture.

However, in Chrome using the Pixel 2 XL in the Developer Tools, if the user starts a panning gesture that is predominantly horizontal, it seems to lock the gesture recognition to the horizontal direction and any change of direction while in flight will not trigger the page to scroll horizontally.

Additionally, in the same Chrome Pixel 2 XL setup, removing the touch-action: pan-y property from the interactive container will trigger a pointercancel event even when panning predominantly in the horizontal direction. This doesn't happen in Safari on iOS 13 since the page cannot scroll horizontally.

This test shows some interoperability issues and I'd like to understand if these are due to bugs in Safari and/or Chrome, or because of some lacking specifications.

Specifically, I have the following questions:

1. Does Chrome apply a directional lock of sorts for scrolling such that starting panning in a predominantly horizontal direction prevents any scrolling in the vertical direction? Cc @NavidZ and @RByers from Google who may know more about this Chrome behavior.

2. If there is such a directional lock, is there a specification that defines this behavior? Under what conditions does that happen? From anecdotal testing, it would appear that, in Chrome, this is tied to the presence of a unidirectional panning restriction using the touch-action property.

3. If there isn't a specification defining this behavior, should there be additional behavior to let page authors enable such directional locks? (As an aside, iOS offers an API on UIScrollView has the directionalLockEnabled property that allows for just that.)

Hopefully all of the Chrome behaviors are already specified, and in this case I'd appreciate some pointers. Otherwise, I think we need to propose some ways to make such content interoperable.

[NavidZ]

Does Chrome apply a directional lock of sorts for scrolling such that starting panning in a predominantly horizontal direction prevents any scrolling in the vertical direction?

Yes. Chrome does do the direction locking for touch scrolling. It is called railing. However, it only holds it if the user doesn't move a substantial amount in other directions. If they do then railing breaks. Here is an example. If you start scrolling horizontally if the finger moves slightly vertically we don't do vertical scroll. But if you move the finger enough in the vertical direction then it starts scrolling in that direction as well. The railing logic has nothing to do with touch-action values though.

If there is such a directional lock, is there a specification that defines this behavior? Under what conditions does that happen? From anecdotal testing, it would appear that, in Chrome, this is tied to the presence of a unidirectional panning restriction using the touch-action property.

As far as I recall there was no specification around the railing logic that Chrome does. However, it is not quite locking either. It is more of a user experience as users are still allowed to scroll in other directions with just a little more movement in the other directions to start it. It is more of a UA behavior to account for the touch sensitivity of screens. Since it is not exactly locking the scroll do you still think a spec is valuable in this space? Also as I mentioned the railing behavior has nothing to do with touch-action.

Note that there is another concept called scroll latching. Sometimes the vertical scroller is the document scroller for example and horizontal scroller is some inner scroller. In that case when we latch to a scroller for the rest of the duration of the scroll only that scroller will scroll regardless of those direction changes (for touch scrolling) and what not. So even if user changes direction if that very scroller that starts the scroll can support scrolling in that new direction (it will break railing) and starts scrolling in that direction. If not just nothing happens. This concept is different from the railing I described earlier. Here is an example you can try. When playing with this example Chrome does latch to a scroller that the scrolling starts on. But doesn't lock the direction of the scroller. It is just that each of the scrollers (one on the document and one on the div with id="scroller") only supports on direction of scrolling. The scroll chains and latching to a scroller is spec'ed here.

[NavidZ]

If there isn't a specification defining this behavior, should there be additional behavior to let page authors enable such directional locks? (As an aside, iOS offers an API on UIScrollView has the directionalLockEnabled property that allows for just that.)

That is the touch-action role here. But touch action doesn't specify quite what the user agent does with that. So I wouldn't say that necessary only applies to the scroll. But we can say here is a new value for touch-action (say 'single-direction') that let the browser to do its own action (being scrolling or pull to refresh or anything else) only on the first direction that user moved their finger. That way UA will lock in the scrolling behaviors and UA will have maybe 3 directions in general like (horizontal, vertical, diagonal) or even maybe only the two x and y directions. Feel free to file a new issue for adding such touch-action value if you believe there are use cases for it out there.

At last there is another note regarding the pointercancel event. Even if the particular scroller that got the scroll latching is not scrollable in the starting direction (this particular case only happens when it is root scroller) and user drags the finger on the page, Chrome still sends a pointercancel as if the user starts moving the finger in the other direction Chrome will starts the scrolling of that latched scroller. Here is an example for that. So if you start scrolling in horizontal direction (even though there is nothing to scroll) but Chrome has already taken over that action and latched to the root scroller. So Chrome sends the pointercancel from the beginning and if user then starts moving the finger in vertical document does get scrolled in the y direction. Does that make sense?

[NavidZ]

Apologies for the previous long comments. I posted them earlier before doing one the last clean up and hence it might have had some clutters in case you are reading them in email.

[graouts]

Thanks for all this info @NavidZ. However, I'm still unclear how my test case gets locked to horizontal scrolling in Chrome. Could you explain what Chrome uses to disable vertical scrolling when an horizontal panning gesture is initiated over the blue area, which is not a scroller but just an area listening to pointer events?

[NavidZ]

Your example falls into the embedded scrollers on the page and the "scroll latching" case I mentioned.

So basically in your page there are two scrollers. The first one is the horizontal scroller which is embedded inside the document vertical scroller. So when you touch the horizontal scroller and move your finger Chrome calculates the scroll chain from where the touch started (basically two scrollers I mentioned) and based on the direction of the scroll it finds the most inner one that can scroll in that direction. From that point on until the finger is released only that scroller will scroll and this never changes for a touch scrolling while touch is down. In that case if the latched scroller is that horizontal scroller then that would scroll until the finger is released regardless of the direction change of scrolling while the finger is down.

[graouts]

Thanks for your patience @NavidZ :)

What I still don't understand is how Chrome determines that the horizontal scroller is a scroller. The scrolling logic is all implemented in JavaScript listening to pointer events, it's just an overflow: hidden container with content that is translated using a transform property that is modified by script.

[graouts]

Removing the touch-action: pan-y property on the .scrollable element really affects Chrome's behavior…

[graouts]

Actually, we can remove JS entirely from the equation, this simple code shows the behavior:

<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<div style="touch-action: pan-y; width: 100%; padding-bottom: 100%; background-color: black;"></div>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>

If you do a horizontal panning gesture over the black rectangle then you will never be able to scroll vertically with this finger in Chrome. If you remove the touch-action value then you can.

[NavidZ]

My bad. I actually did build all those examples of mine based on your example but when replying to your last comment I just tried it on the phone and forgot what the code was doing. You are right that there is indeed no second scroller. Only the main document vertical scroller.

So Chrome in general decides to think about scrolling when touch point moves beyond slop region. At that point Chrome knows the (at least initial) direction of scrolling. Based on that it first looks at the touch-action. If it is allowed then it sends a pointercancel and start looking into scroll chain and finding the latched scroller for that direction and scrolling. If it is not allowed it gives up the scrolling altogether for the rest of the time it is on the screen regardless of which direction it moves. I believe we followed this text from the spec although it doesn't call this touch direction changes explicitly:

"Once a touch action has been started, and the user agent has already determined whether or not the action should be handled as a user agent touch behavior, any changes to the relevant touch-action value will be ignored for the duration of the touch action"

However, I cannot think of a scenario now that why we cannot start scrolling if the direction really changes and goes in a direction that touch-action allows. Then I assume we would need to send a pointercancel. Although I need to think about this a little more to make sure whether there is anything going wrong as a result of that.

[graouts]

"Once a touch action has been started, and the user agent has already determined whether or not the action should be handled as a user agent touch behavior, any changes to the relevant touch-action value will be ignored for the duration of the touch action"

My interpretation of this paragraph does not relate to the issue here. To me, what this means is that a change to the touch-action value has no bearing during a given touch gesture.

I think we have two options:

1. Specify in Pointer Events that if a single panning direction is specified with the touch-action property that initially panning in a contrary direction should prevent scrolling in the direction specified by touch-action. This is the current Chrome behavior and I expect we will implement this in WebKit shortly.

2. Specify another CSS property or extend the definition of an existing CSS property to specify whether a fragment may have directional locking for scrolling.

At any rate I believe we must specify something since the spec as-is is at best unclear but in my opinion does not sufficiently define user-agent behavior for such interactions causing interoperability issues for what I believe to be a reasonable design as demonstrated in the test.

[patrickhlauke]

Discussed on PEWG call today https://www.w3.org/2020/07/22-pointerevents-minutes.html#item02 - @NavidZ and @mustaqahmed to sync up and likely propose some note/wording for PE spec (option 1 from @graouts)

[NavidZ]

As @graouts mentioned his understanding of that paragraph is correct and that is normative. But maybe a note to give the same example as @graouts mentioned in his option 1 would be nice.

[lincolnthree]

Just joining this thread (as an end-developer). I've attempted to come up to speed on this topic, so please forgive me if I get any assumptions wrong. But I had a thought this morning that might be useful.

It seems like some type of spec'd "scroll-track-locking" API might be useful to control the track direction browsers allow, specifically, once a touch/scroll event has already been started.

This linked issue: ionic-team/ionic-framework#21193 shows an example where scrolling may be desired in one direction, but it may still be useful for advanced gestures to allow scrolling in another direction, and to allow that change to occur programmatically (or via CSS, etc).

E.g. Consider a scenario where a users starts scrolling an element (like a swiper) horizontally, but reaches an arbitrary boundary
where they can then interact on the vertical axis. Locking horizontal scroll, and switching to vertical scroll could be a useful pattern.

I don't know if "on the fly directional lock", after an event/events have been started, is something being considered, but thought I'd throw it into the mix as something that could benefit from spec'd behavior. Since apparently some browsers (as I gather from reading) handle this differently.

[patrickhlauke]

Circling back to the idea of adding a note in the spec, a la

Specify in Pointer Events that if a single panning direction is specified with the touch-action property that initially panning in a contrary direction should prevent scrolling in the direction specified by touch-action. This is the current Chrome behavior and I expect we will implement this in WebKit shortly.

still wondering - do we want to make this mandatory behavior? Or an non-normative note, couched in "user agents may choose to...", i.e. should it be left up to implementations (though it can cause interop issues)?

[lincolnthree]

@patrickhlauke I'm just an observer, but my inclination is to lean toward mandatory behavior. As you mentioned, inter-op issues are a concern of option 2. If exceptions need to be made, perhaps additional customization/flexibility can be added in once more use-cases or issues with the design become apparent?

[mustaqahmed]

I just realized that we have the following WPTs to enforce railing behavior already, with the assumption that railing is active only when panning has started (instead of when finger has moved significantly):
pointerevents/pointerevent_touch-action*.html

My bad that I missed this important point before! Reopening.

[mustaqahmed]

Here is a more precise observation about existing tests, we need to consolidate our discussion here with this: this WPT, for example, has touch-action: pan-x and expects a down-then-right swipe gesture to pan the page in the second half of the swipe.

[mustaqahmed]

I have just completed my investigation, and concluded that I misrepresented those WPTs in my two recent comments 🤦🏼‍♂️!

... expects a down-then-right swipe gesture to pan the page in the second half of the swipe

Correct interpretation: the test injects a "down" gesture followed by a separate "right" gesture. Same with other similar tests: they inject separate gestures, and never an L-shaped gesture. So the tests are working as intended.

My conclusion:

1. The spec is fine today because Sec 8.2 talks about single-finger interaction only. Developers still need to abort their custom gesture handling on pointercancel because a single-finger interaction can turn into a two-finger one at any moment. I will leave this issue open in case this needs to be called out as a note or something.

2. Both Chrome and Firefox show the railing behavior we discussed earlier here, i.e., the first move outside slop determines the pan direction (as long as it remains a single-finger interaction of course). Therefore, @patrickhlauke's touch-action-scrolling demo on pan-x works exactly how @flackr argued today with his developer's hat on. Safari doesn't have railing behavior, so any little left-right movement kicks off a browser panning action.

3. Existing WPTs are all fine. We need new WPTs for zigzag gestures.

4. The pointercancel event seems to be the only guaranteed signal for developers to detect the browser taking over control, because of the possibility of one- to two-finger transition mentioned above. Along the same line, I also confirmed that the browser may fire a pointermove way outside the slop region before the pointercancel: a fast flick here on touch-action: auto easily causes a pointermove event as far as 100px off from the pointerdown.

[smaug----]

So we can close this?

[mustaqahmed]

Working on a test now.