Add a third drag speed to KeyboardDragListener

[pixelzoom]

In phetsims/faradays-electromagnetic-lab#54 (comment), there is a request to have 3 speeds for moving Nodes via the keyboard. KeyboardDragListener currently only supports 2 speeds, via options dragVelocity and shiftDragVelocity.

AccessibleValueHandler (e.g. Slider) supports 3 drag speeds via options keyboardStep, shiftKeyboardStep, and pageKeyboardStep. So there is prescedent for 3 speeds.

Questions:
(1) Would it be possible to add a third drag speed to KeyboardDragListener? Estimate of effort?
(2) Using the same naming schema as AccessibleValueHandler, should it be named pageDragVelocity, and by convention be faster than dragVelocity?
(3) Would "page" work for 2D dragging, which requires all 4 arrow keys, or would some other modifer be more usable?

I'll start by assigning to the authors of KeyboardDragListener.

@arouinfar @kathy-phet @ariel-phet FYI.

[pixelzoom]

In #1603, it's noted that dragVelocity and shiftDragVelocity are incorrectly named. Velocity is a vector; speed is a scalar. So the options for drag speed should have "dragSpeed" in their names, not "dragVelocity".

[zepumph]

I also like renaming to "speed".

I'm very happy to talk through adding a third possible speed to keyboard drag listener. That seems like an improvement, especially if there is a sim that is desiring that design. I'm a bit confused about what the interaction would be for the third speed. Yes there is a third case in sliders where you can jump in ~1/10 the range with page keys, but as you said that doesn't map too well into 2-d dragging. We have arrow keys and shift+arrow keys, but what would the third speed be triggered by? Holding pageup/down as a modifier key to the arrow keys doesn't really make a lot of sense to me. Perhaps ctrl+arrow keys, but I have no idea about the ramifications of that. To me it is a question for the designers of FEL. I don't think it is on @jessegreenberg and I to decide on what the keys should be.

In terms of the effort of adding this feature. I don't think it a quick and dirty addition of the feature would be too hard as it would be easy enough to follow the pattern used by the shiftDragVelocity/Delta, but here are some questions that come up:

1. I wish that Re-implement KeyboardDragListener with KeyboardListener #1570 could happen before we continue to work on and improve the current KeyboardDragListener implementation.
2. We have two different apis in KeyboardDragListener speed (aka velocity) and delta (per key press instead of time-based). We would likely want to support this third speed for both cases as well.

[jessegreenberg]

Adding a third speed for KeyboardDragListener seems reasonable. As described above we should identify which modifier key we should use. "Page" isn't a modifier key it is a specific key press often assigned to arrow keys when the function key is pressed. @pixelzoom will follow up with the design team.

[pixelzoom]

I asked about accessibility guidelines/conventions for use of modifier keys at dev meeting today, and no one was aware of general guideline. Shift was apparently used in AccessibleSlider for "smaller increment" because that the HTML convention for something with role "slider".

I investigated what Microsoft, Apple, and ARIA have to say about modifier keys.

Microsoft's Guidelines for Keyboard User Interface Design says this about modifier keys:

...

• Use a shortcut with the CTRL key for actions that represent a large-scale effect, such as CTRL+S for save current document.
• Use the SHIFT+ key combination for actions that extend or complement the actions of the standard shortcut key. For example, the ALT+TAB shortcut key displays the primary window of a running application. Alternatively, the SHIFT+ALT+TAB key combination allows you to navigate backward through currently running applications that have been previously accessed.
  ...

Apple's macOS HIG has this to say about modifier keys in design for Keyboards:

...
Let people use modifier and other keys to adjust the behavior of some interactions. For example, pressing Command while dragging moves items as a group, and pressing Shift while drag-resizing constrains resizing to the item’s aspect ratio. In addition, holding an arrow key moves the selected item by the smallest app-defined unit of distance until people release the key.
...
Prefer the Command key as the main modifier key in a custom keyboard shortcut. Most standard keyboard shortcuts use the Command key, so people are familiar with it.
Prefer the Shift key as a secondary modifier when the shortcut complements another shortcut. For example, Command-P displays the Print dialog in most apps. The standard shortcut for the Page Setup dialog, which complements printing, is typically Shift-Command-P.
Use the Option modifier sparingly. For example, you might use Option in the shortcut for a less-common command or a convenience or power feature. For example, the Finder uses Option-Command-W for Close All Windows and Option-Command-M for Minimize All Windows.
As much as possible, avoid using the Control key as a modifier. The system uses Control in many systemwide features, like moving focus or capturing screenshots.
...

That last point for macOS is worth noting -- Apple recommends avoiding the Control modifier.

Finally... ARIA's Best Practices for Developing a Keyboard Interface has no relevant recommendations for the use of modifier keys. It only describes the focus behavior of Shift+Tab, and includes a table of key assignments for common function.

[pixelzoom]

Next step is to identify the desired modifier key, to be used with the arrow keys for "drag faster". I did some research (see the previous comment) and came to no conclusion. @zepumph and @jessegreenberg felt that the choice of key should be a designer's call. Since we'd like this feature in FEL, I'll assign to @arouinfar.

[arouinfar]

Is it possible to use ctrl for Windows and cmd on Mac? These are keys that are frequently used in shortcuts, so users likely have some muscle memory when it comes to using them, even if they don't map perfectly to the Windows/Apple guidelines.

I would like to avoid multi-key modifiers as much as possible, so something like page up/down won't work because keys are accessed with fn+left/right on most laptops. Maybe fn by itself would work? It's a standard key on Macbooks, but not sure about how that would work in a Windows environment.

[pixelzoom]

I'm not familiar with what has been done in other sims (or common code) so I can't recommend a modifier key for this. @jessegreenberg or @zepumph are probably more familiar with prior work. What do you suggest for the "drag faster" modifier key?

[zepumph]

I do not know if we have any precedent in the project for using "ctrl" on windows and "cmd" on mac, even though that is clearly a reasonable path. In fact, in studio we awkwardly are using just the ctrl, which I believe is awkward on a mac since it isn't the cmd key:

https://github.com/phetsims/studio/blob/ea0c18b2a74ae86335388ae3d3a24ef022a099ed/js/Select.ts#L99-L115

I'll let @jessegreenberg weigh in a bit more.

[jessegreenberg]

using "ctrl" on windows and "cmd" on mac,

We do that for zoom commands to mimic default browser behavior. But "cmd + arrow keys" is the native Mac command to go forward/backward a page. If we use that here we would have to prevent that default behavior.

Do we need shift to always make dragging slower? We could open it up to any speed if some sims just need a faster modifier.
I think alt on windows/option for mac could also work well.

@terracoda do you have any opinions about this issue?

[terracoda]

I am not opposed to adding a third speed, but we originally chose the shift key as a second option for step size because that's how sliders work. We can't port pageUp/Down as easily because these keys do not exist on every keyboard.

I am wondering if what we want is an option for more than one additional speed rather than a random new modifier key.

[pixelzoom]

Found it - phetsims/phet-info#211
We need to create a list of our current hot keys.

I very much agree! This has been a problem every time I've need to add hotkeys to a sim. And we've been kicking that can down the road for almost 2 years. phetsims/phet-info#211 was broken out of #1445, which was broken out of phetsims/phet-info#188 where I said:

There's currently no "global shortcuts" list, and it's impractical to expect code reviewers (or even the sim developer) to search the code. So a list would be useful during development, to prevent collisions in the first place.

[pixelzoom]

To answer a couple of @terracoda's questions:

Is KeyboardDragListener what you use for custom objects like the Bar Magnet in FL and the Balloons in BASE?

Yes.

How are the needs for KeyboardDragListener the same or different than that of AccessibleValueHandler?

KeyboardDragListener changes a position, which is a vector. It's also responsible for constraining that position to optional drag bounds (a rectangle), and transforming the position between model and view coordinate frames. It is implemented as a class.

AccessibleValueHandler changes a number (scalar). It is implemented as a trait that can be added to a class (like Slider).

[pixelzoom]

@terracoda asked:

What is the implementation of the Bar Magnet in FL?

I'm not sure if I understand the question, so ask again if I'm not answering it. If you're asking what the FEL bar magnet is using for alt input... All draggable objects (including the bar magnet) are using KeyboardDragListener, with values for dragSpeed and shiftDragSpeed still to be fine-tuned in phetsims/faradays-electromagnetic-lab#54 (comment).

[terracoda]

Ok, I need to retract a couple of things I said.

1. It seems using the Shift key for a slower speed or smaller steps is a PhET thing - something we decided we wanted for BASE's balloons. It's not something that we got from how sliders work by default.

2. The Shift key is always for a slower speed, i.e. for smaller steps.

Given that, I feel the f-key is a good modifier for a "faster" quasimode, i.e. bigger steps. The f-key is a marked home key on QWERTY keyboards, and is a good mnemonic, at least in the English language, for "faster" speed or faster/bigger steps.

It might be nice to have a bit of design meeting time to openly discuss this idea.

Implementing the f-key would not exclude using keys like 1-2-3 for interactions where experimenting with controlled velocities is important, like in FL and FEL.

[terracoda]

Thanks @pixelzoom, for your explanation in #1601 (comment)
It is good to know that all draggable things (that are NOT sliders, that is) use KeyboardDragListener.

Edited to add "NOT".

[terracoda]

So to summarize, I am proposing we add the f-key as a third speed, so KeyboardDragListener can have:

• a default speed or step size
• a slower speed or smaller step size with [Shift] + Arrow key
• a faster speed or bigger step size with [F] + Arrow key

[pixelzoom]

• a faster speed or bigger step size with [F] + Arrow key

I'm familiar with using letters as shortcuts -- Sasha Maximova's article that you referred to recommends letters as a way of avoiding OS/browser conflicts. But I'm not familiar with using letters as modifier keys, which are typically held down. Is there a precedent for that?

[arouinfar]

I am fine with @terracoda's proposal to use the F-key as a modifier for fast speed.

That said, this feature is not required for FEL. The design team thought it could be a useful feature, broadly speaking, so we asked @pixelzoom to create this issue. However, the learning goals for FEL are satisfied with the 2 existing drag speeds in KeyboardDragListener.

[terracoda]

@pixelzoom, using letter keys as modifiers is not new. I did it my first sim, Balloons and Static Electricity, to avoid conflicts with screen reader software shortcuts.

Also Sasha Maximova suggests pairing letter keys as well.

For us we’ll be pairing f with arrow keys and f with WASD keys.

[pixelzoom]

Apologies for continuing this discussion, but trying to educate myself about conventions for modifier keys.

@terracoda said:

Also Sasha Maximova suggests pairing letter keys as well.

For us we’ll be pairing f with arrow keys and f with WASD keys.

Can you elaborate on what you mean by "pairing"? Will 'f' be held down, while arrow/WASD keys are pressed. If not, that will be different that Shift. If so, it does not seem to match Sasha's recommendations for arrow keys.

I've read J, K, or How to choose keyboard shortcuts for web applications as you recommended. Can you point me to where Sasha suggests using a letter as a modifier, to be held down while a second key is pressed?

What I find in the article is quite the opposite -- she recommends that letter keys should NOT be held down, but should be pressed/released in a sequence, with a delay between them. For two letter shortcuts, the first letter should access the feature/action, while the second letter should be the restriction (modifier) of that feature/action.

Combining one and two letter shortcuts

It’s better not to use both one and two letter shortcuts if they start with the same letter. For instance, you want to use c for comments, and cr for comment restrictions. The first keystroke will bring up the first action. Technically, you can add a delay after the first letter, to wait for a (probable) second key press.

How does our proposed use of f-arrow and f-WASD match Sasha's recommendation?

[terracoda]

Thanks @pixelzoom, good point, but the examples in the articles for 2 consecutive key presses are for a completely different category of interaction. Pressing a letter to bring up a comment box or menu, then doing something else.

We need an intuitive way to move an object at a different speed or step size in any direction.

I am building off my research and design experience with BASE, and with what we already have for the slower speed.
I actually tried a modal approach with consecutive key presses in interviews with blind learners. JAWS has a by-pass key, so we explored using the by-pass key then the jump location key in BASE and it was NOT AT ALL intuitive.

I think I even tried j-key, then location key, but I do not remember at this point.

For BASE, the j-key as a modifier key to enter a quasimode for jumping and adding a second key for the location of the jump was VERY intuitive and memorable for learners using BASE.

Example from BASE for reference:

Now for custom dragging...
Since we already have a quasimode for slow, Shift + Arrow key, I was suggesting we also have a quasimode for fast, and use the f-key as a modifier key to enter that quasimode.

A quasimode is harder for people with limited dexterity, that's why the 1-2-3 shortcuts are really handy for certain interaction scenarios, like the Bar Magnet in FL.

Using the f-key as modifier rather than a single shortcut to make a bigger step should avoid conflicts with other software that might use the f-key as some other shortcut.

Quoting from the article:

There are hardly any standards in this field, but if we wish to flatten the learning curve for our customers, we must consider current conventions.

I feel, the j-key in BASE started a PhET convention.

The article also says a lot of other things like documentation is important. That is currently lacking and part of another open issue.

If we need to discuss this more, we should do it in design meeting tomorrow.

[pixelzoom]

Thanks for the clarification @terracoda.

In #1601 (comment), @arouinfar said:

That said, this feature is not required for FEL. The design team thought it could be a useful feature, broadly speaking, so we asked @pixelzoom to create this issue. However, the learning goals for FEL are satisfied with the 2 existing drag speeds in KeyboardDragListener.

@arouinfar and I discussed this issue in our standup meeting this morning. Since this feature is not needed for FEL, or any other sim in development, we're going to label this issue as deferred. If another team needs this feature, they can resume the discussion and decide how/whether to implement.

[zepumph]

Given how this issue had great discussion, and a reasonable potential path forward, but wasn't adopted, I think the best path forward is to close this issue. When there is a need for a third speed. We will still be able to find this issue and recall this discussion. To me it comes down to the value added vs the challenge to learn a new input combo. I think it is unlikely that a third drag speed will be that valuable in the near future.