Keyboard redesign

[keaganeng]

Redesign of Doenet's keyboard based on GeoGebra and professor feedback.

[keaganeng]

Questions that still need to be answered: @djnykamp @kevincharles @jaltekruse

1. Biggest issue: the keys that have a right parenthesis/bracket have an issue where they automatically include the left parenthesis/bracket and not the right parenthesis/bracket, unless the input field is unfocused. When you try to close the closure by clicking the right parenthesis, it creates another inner closure instead of closing. What should we do about this? There are quite a few keys with this issue. I recommend you test it out yourself to see what I'm talking about. I also included a demo here.
2. On the Greek keyboard, do we want phi (Φ) in two places? GeoGebra's keyboard does, and that's what I've based the design from.
3. Has the negate key on the symbols keyboard been coded into core?
4. Does the behavior of the \int key on the f(x) keyboard look okay?

Screen.Recording.2023-03-06.at.5.38.40.PM.mov

[jaltekruse]

Questions that still need to be answered: @djnykamp @kevincharles @jaltekruse

1. Biggest issue: the keys that have a right parenthesis/bracket have an issue where they automatically include the left parenthesis/bracket and not the right parenthesis/bracket, unless the input field is unfocused. When you try to close the closure by clicking the right parenthesis, it creates another inner closure instead of closing. What should we do about this? There are quite a few keys with this issue. I recommend you test it out yourself to see what I'm talking about. I also included a demo here.

@keaganeng I'm pretty sure the issue with the parens should be solvable by using "type )" instead of "cmd )". and the same should be done for [] and {}.

Update: This was wrong, "cmd )" and "type )" don't behave differently for these cases. I believe I tested things out on desmos, who have a nice UX for mathquill and their virtual keyboard for the parens and such and then just assumed the Free Math keyboard (one I have access to source code for) worked similarly, but in fact Free Math is currently broken in the same ways that doenet prod is. I spent time with Keagan today and figured out what we actually needed to do to make this work. The biggest thing was how clicking on the keyboard was making the mathquill lose focus, which was auto-accepting any ghost parens/brackets/etc. We now have it working so that typing on they virtual keyboard prevents the mathquill (and more specifically the internal matharea mathquill uses to capture keystroke events) from losing focus ever, so users can now type just like they can on the desmos site, including mismatched parens and brackets for interval notation, and closing a paren or bracket early (the whole point of drawing a ghost instead of instantly drawing a "full" paren).

This does appear to be an issue with other things that close, but don't appear on physical keyboards as a single key, like \rangle. I see in the version before this that we had a workaround of "keystroke Right" for close paren. This isn't perfect, but does act better than cmd \rangle. The only issue I see with it is closing a \rangle earlier than where the ghost is currently appearing won't work. i.e. if the vertical bar is the cursor \langle 5 + 7 | - 10 \ghostrangle, would move the cursor rather than close it at the current cursor position. Video attached too.

(The fix that Keagan and I made today also handles this more advanced case where you want to close a ghost paren "early", also the workaround discussed next also is unnecessary)

actually as I was recording the video I found that when it moved the cursor the the right it also happens to close the rangle, so this might be fine. And we might be able to make it "seamless" to users with issuing both a "keystroke Right" and a "keystroke Left" to get the cursor back in the same place

Edit: I was curious if we were handling all cases with the ghosting logic, here is where it is implemented in mathquill, I'm pretty sure this is just saying it works for () [] {} \langle \rangle || for absolute value and unicode or latex synonyms of these things. https://github.com/mathquill/mathquill/blob/ead472c3c3b835a1ed21d0ac11611fae195dfdb8/src/commands/math/commands.js#L731

close_rangle-2023-03-06_18.17.22.mp4

[dqnykamp]

See #1750 for some earlier discussion on the closing parens problem. I don't know how to fix it, as I tried different commands and they didn't seem to work. It works fine on the Desmos keyboard, though. Can we copy what they did or does it depend on mathQuill features that aren't integrated into the main mathQuill repository?

[dqnykamp]

2. On the Greek keyboard, do we want phi (Φ) in two places? GeoGebra's keyboard does, and that's what I've based the design from.

One of the keys should be \phi, the other should be \varphi. They are currently translated to the same thing by Doenet, but we discuss if that should change.

[dqnykamp]

3. Has the negate key on the symbols keyboard been coded into core?

We are not planning to include the negate symbol. Instead, we are interested in experimenting with the idea that the "negate" key acts like the shift key. It would change subset to not subset, etc. However, it @jaltekruse mentioned that mathQuill might not implement all these negated symbols, in which case we might need to rethink what to do for now.

[dqnykamp]

4. Does the behavior of the \int key on the f(x) keyboard look okay?

I don't think we can currently get the correct behavior of the \int key due to a bug in mathQuill. It seems to want to always have the super/subscripts like the definite integral \int_a^b. I think the current behavior of \int is what we want for \int_a^b. Since the correct behavior for \int is currently impossible with mathQuill, we may want to keep if off the keyboard for now. I don't know what it would take to fix mathQuill to allow it to represent indefinite integrals.

[dqnykamp]

Can you merge upstream/main into this branch so that we can test it with the new math features built into core? For example, the nPr, nCr buttons don't work correctly (they seem to enter n*Cr, for example), but I hope that would be fixed with the new updates. Also, they shouldn't show up as italics in the keyboard. Can you use \operatorname{nCr} for what shows up on the keyboard itself?

[keaganeng]

Can you merge upstream/main into this branch so that we can test it with the new math features built into core? For example, the nPr, nCr buttons don't work correctly (they seem to enter n*Cr, for example), but I hope that would be fixed with the new updates. Also, they shouldn't show up as italics in the keyboard. Can you use \operatorname{nCr} for what shows up on the keyboard itself?

Yes, this seems to be fixed now:

[keaganeng]

One of the keys should be \phi, the other should be \varphi. They are currently translated to the same thing by Doenet, but we discuss if that should change.

This is fixed now!

[keaganeng]

We are not planning to include the negate symbol. Instead, we are interested in experimenting with the idea that the "negate" key acts like the shift key. It would change subset to not subset, etc. However, it @jaltekruse mentioned that mathQuill might not implement all these negated symbols, in which case we might need to rethink what to do for now.

So, should I remove the negate key from the symbols keyboard, or leave it as it is for now? If we decide to remove it, what would take its place?

[keaganeng]

I don't think we can currently get the correct behavior of the \int key due to a bug in mathQuill. It seems to want to always have the super/subscripts like the definite integral \int_a^b. I think the current behavior of \int is what we want for \int_a^b. Since the correct behavior for \int is currently impossible with mathQuill, we may want to keep if off the keyboard for now. I don't know what it would take to fix mathQuill to allow it to represent indefinite integrals.

Should we remove the \int key from the f(x) keyboard, and if we decide to remove it, what should replace it?

[keaganeng]

Also, these keys seem to not be typing anything, but I'm wondering if I made a mistake in the code?

[dqnykamp]

I don't have an answer on the questions about the negate or int keys. Maybe we can discuss in the authoring group on Thursday.

[dqnykamp]

It looks like mathQuill doesn't support entering \mathbb. However, you can type \R, \Q, \Z, \N, and \C, which turn into the \mathbb versions.

I haven't added support to \mathbb to math-expressions, so Doenet won't know how to handle these yet. I have to think about the best approach.

[jaltekruse]

Thinking about this a little more, we probably don't want this code to throw an exception if we happen to ever end up with a page that has a MathInput and no keyboard (we likely always want the keyboard to always appear with math inputs, but a silent exception here isn't the right way to enforce that), I would change the if to something like keyboard && keyboard.contains...

[dqnykamp]

Right, it turns out this crashes the test page and the cypress test page, as they don't have the keyboard. I'll make a new PR changing it to keyboard?.contains.