Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Windows touch keyboard - special characters not working in WSL #10308

Closed
leoshusar opened this issue Jun 2, 2021 · 8 comments
Closed

Windows touch keyboard - special characters not working in WSL #10308

leoshusar opened this issue Jun 2, 2021 · 8 comments
Labels
Area-Input Related to input processing (key presses, mouse, etc.) Resolution-Duplicate There's another issue on the tracker that's pretty much the same thing.

Comments

@leoshusar
Copy link

Windows Terminal version (or Windows build number)

Microsoft Windows [10.0.19043.1023]

Other Software

No response

Steps to reproduce

  • open WSL 2
  • open touch keyboard
  • type any "AltGr char" (I tried for example @, $, €, [, ], ^ on Czech keyboard or € on English keyboard)

Expected Behavior

Those characters should work normally

Actual Behavior

When I write those characters, they won't appear in WSL and Windows "error" sound plays.
One difference I found is when you write # char, it writes it and sends "return" (or newline) after it.

It doesn't matter if I open WSL in Terminal, Console, or if I open PowerShell and then run wsl command. Those AltGr chars just stops working.
@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Jun 2, 2021
@leoshusar leoshusar changed the title Windows touch keyboard - special characters not working Windows touch keyboard - special characters not working in WSL Jun 2, 2021
@zadjii-msft
Copy link
Member

Out of curiosity, does this repro with the "On Screen Keyboard", or only the "Touch Keyboard"? Shockingly, those are two totally different codebases 😖

This sounds a lot like #10551, which I'm pretty confident is fixed in Windows 11. I'd bet that the Touch Keyboard sends garbage vkeys, or garbage scan codes, or something of the like.

@zadjii-msft zadjii-msft added Area-Input Related to input processing (key presses, mouse, etc.) Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something labels Jul 6, 2021
@leoshusar
Copy link
Author

leoshusar commented Jul 6, 2021 •
edited
Loading

Just tried it and happens with both keyboards.
Is there anything I can try? Any way to see what is being sent to WSL?

EDIT: I found showkey linux command which prints codes sent by keyboard.
I tried AltGr + @, then & and # and got this: ^@^[@^@, ^@^[&^@, ^@^[#^@
^@ is being sent on AltGr key toggle, so I guess it's AltGr press (^@), then & key press (^[&), AltGr unpress (^@).

When I send those keys from physical keyboard, only characters show up in showkey.

Screen here

@ghost ghost added Needs-Attention The core contributors need to come back around and look at this ASAP. and removed Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something labels Jul 6, 2021
@zadjii-msft
Copy link
Member

The following might seem convoluted, but it might be able to help us figure out what the key getting sent to the terminal is:

  1. enable debug features by adding "debugFeatures": true, to the global settings
  2. Hold both the left and right alt keys, and open a new tab. What you should see is a tab with two panes in it, and the second one will have the "debug tap", showing all the input and output sent to the Terminal, something like:
    image
  3. repro the issue
  4. copy the output in the second pane to this issue, (and maybe share a screenshot so we can differentiate between the output and input (the red text)

@leoshusar
Copy link
Author

leoshusar commented Jul 8, 2021 •
edited
Loading

Ok, I tried three variants. In all three I typed @&# with AltGr release after each key.

Physical keyboard

raw data 
␛[17;29;0;1;8;1_␛[18;56;0;1;265;1_␛[86;47;64;1;9;1_␛[86;47;64;0;9;1_@␛[17;29;0;0;1;1_␛[18;56;0;0;0;1_␛[17;29;0;1;8;1_␛[18;56;0;1;265;1_␛[67;46;38;1;9;1_␛[67;46;38;0;9;1_&␛[17;29;0;0;1;1_␛[18;56;0;0;0;1_␛[17;29;0;1;8;1_␛[18;56;0;1;265;1_␛[88;45;35;1;9;1_␛[88;45;35;0;9;1_#␛[17;29;0;0;1;1_␛[18;56;0;0;0;1_
screen

physical

On-Screen Keyboard

raw data 
␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[86;47;64;1;11;1_␛[86;47;64;0;11;1_␇␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[67;46;38;1;11;1_␛[67;46;38;0;11;1_␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[88;45;35;1;11;1_␛[88;45;35;0;11;1_#␛[32m␛[1m␍␊leoshusar@leoshusar-pc␛[m:␛[34m␛[1m/mnt/c/Users/husar␛[m$␣␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_
screen

on-screen

Touch Keyboard

raw data 
␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[86;47;64;1;11;1_␛[86;47;64;0;11;1_␇␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[67;46;38;1;11;1_␛[67;46;38;0;11;1_␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[88;45;35;1;11;1_␛[88;45;35;0;11;1_#␍␊␛[32m␛[1mleoshusar@leoshusar-pc␛[m:␛[34m␛[1m/mnt/c/Users/husar␛[m$␣␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_
screen

touch

EDIT: recreated physical test for better comparison

EDIT2: I also tried PowerShell, just physical vs Touch Keyboard

Physical

raw data 
␛[17;29;0;1;8;1_␛[18;56;0;1;265;1_␛[86;47;64;1;9;1_␛[86;47;64;0;9;1_␛[?25l␛[91m␛[7;18H>␣␛[93m@␛[?25h␛[17;29;0;0;1;1_␛[18;56;0;0;0;1_␛[17;29;0;1;8;1_␛[18;56;0;1;265;1_␛[67;46;38;1;9;1_␛[67;46;38;0;9;1_␛[m␛[?25l␛[93m␈@␛[m&␛[?25h␛[17;29;0;0;1;1_␛[18;56;0;0;0;1_␛[17;29;0;1;8;1_␛[18;56;0;1;265;1_␛[88;45;35;1;9;1_␛[88;45;35;0;9;1_␛[?25l␛[93m␛[7;20H@␛[m&␛[32m#␛[?25h␛[17;29;0;0;1;1_␛[18;56;0;0;0;1_
screen

physical

Touch Keyboard

raw data 
␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[86;47;64;1;11;1_␛[86;47;64;0;11;1_␛[?25l␛[91m␛[7;18H>␣␛[93m@␛[?25h␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[67;46;38;1;11;1_␛[67;46;38;0;11;1_␛[m␛[?25l␛[93m␈@␛[m&␛[?25h␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_␛[17;29;0;1;8;1_␛[18;56;0;1;10;1_␛[88;45;35;1;11;1_␛[88;45;35;0;11;1_␛[?25l␛[93m␛[7;20H@␛[m&␛[32m#␛[?25h␛[17;29;0;0;3;1_␛[18;56;0;0;1;1_
screen

touch

@leoshusar
Copy link
Author

So, just came to say that after upgrade to Win11 it indeed works fine.

@zadjii-msft
Copy link
Member

Well that's good to hear! Let's just call this /dup #10551 until we're proven otherwise then. Thanks!

@ghost
Copy link

ghost commented Oct 15, 2021

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

@ghost ghost closed this as completed Oct 15, 2021
@ghost ghost added Resolution-Duplicate There's another issue on the tracker that's pretty much the same thing. and removed Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements Needs-Attention The core contributors need to come back around and look at this ASAP. labels Oct 15, 2021
@j4james
Copy link
Collaborator

j4james commented Dec 30, 2023 •
edited
Loading

FYI, the reason this doesn't work is because the Touch and On-Screen keyboards generate too many Alt modifiers in the control key state. On a physical keyboard, when you press AltGr, it actually generates two keypresses - a LeftCtrl and a RightAlt. This is because there isn't actually an AltGr mapping in Windows keyboard layouts - it's really just a Ctrl+Alt mapping, so the AltGr key needs to emulate that state.

However, when you press AltGr on a touch keyboard, it generates a LeftCtrl and a LeftAlt. That would be fine, except when the character key is pressed, it also sets an additional RightAlt modifier for no apparent reason. So we'll see the key as something like Ctrl+LAlt+RAlt+4, which is interpreted as Alt+AltGr+4, and thus you get an unexpected ESC prefix on your € character.

I've got a fix for this is in my keyboard refactor for #16506 (PR #16511), which I think should still be harmless on versions of Windows that don't have the bug. The way it works is I keep a record of the last control key state, and if I receive a key press with a new RightAlt modifier set, when it wasn't a RightAlt key that was pressed, I simply discard that modifier. It's a bit of a hack, but it works.

@j4james j4james mentioned this issue Dec 31, 2023
Merged
12 tasks
DHowett pushed a commit that referenced this issue Jan 30, 2024
@j4james
Refactor VT terminal input (#16511)
dccc1f4 
The primary reason for this refactoring was to simplify the management
of VT input sequences that vary depending on modes, adding support for
the missing application keypad sequences, and preparing the way for
future extensions like `S8C1T`.

However, it also includes fixes for a number of keyboard related bugs,
including a variety of missing or incorrect mappings for the `Ctrl` and
`Ctrl`+`Alt` key combinations, 

## References and Relevant Issues

This PR also includes a fix for #10308, which was previously closed as a
duplicate of #10551. I don't think those bugs were related, though, and
although they're both supposed to be fixed in Windows 11, this PR fixes
the issue in Windows 10.

## Detailed Description of the Pull Request / Additional comments

The way the input now works, there's a single keyboard map that takes a
virtual key code combined with `Ctrl`, `Alt`, and `Shift` modifier bits
as the lookup key, and the expected VT input sequence as the value. This
map is initially constructed at startup, and then regenerated whenever a
keyboard mode is changed.

This map takes care of the cursor keys, editing keys, function keys, and
keys like `BkSp` and `Return` which can be affected by mode changes. The
remaining "graphic" key combinations are determined manually at the time
of input.

The order of precedence looks like this:

1. If the virtual key is `0` or `VK_PACKET`, it's considered to be a
   synthesized keyboard event, and the `UnicodeChar` value is used
   exactly as given.

2. If it's a numeric keypad key, and `Alt` is pressed (but not `Ctrl`),
   then it's assumedly part of an Alt-Numpad composition, so the key
   press is ignored (the generated character will be transmitted when
   the `Alt` is released).

3. If the virtual key combined with modifier bits is found in the key
   map described above, then the matched escape sequence will be used
   used as the output.

4. If a `UnicodeChar` value has been provided, that will be used as the
   output, but possibly with additional Ctrl and Alt modifiers applied:

   a. If it's an `AltGr` key, and we've got either two `Ctrl` keys
      pressed or a left `Ctrl` key that is distinctly separate from a
      right `Alt` key, then we will try and convert the character into
      a C0 control code.

   b. If an `Alt` key is pressed (or in the case of an `AltGr` value,
      both `Alt` keys are pressed), then we will convert it into an
      Alt-key sequence by prefixing the character with an `ESC`.

5. If we don't have a `UnicodeChar`, we'll use the `ToUnicodeEx` API to
   check whether the current keyboard state reflects a dead key, and if
   so, return nothing.

6. Otherwise we'll make another `ToUnicodeEx` call but with any `Ctrl`
   and `Alt` modifiers removed from the state to determine the base key
   value. Once we have that, we can apply the modifiers ourself.

   a. If the `Ctrl` key is pressed, we'll try and convert the base value
      into a C0 control code. But if we can't do that, we'll try again
      with the virtual key code (if it's alphanumeric) as a fallback.

   b. If the `Alt` key is pressed, we'll convert the base value (or
      control code value) into an Alt-key sequence by prefixing it with
      an `ESC`.

For step 4-a, we determine whether the left `Ctrl` key is distinctly
separate from the right `Alt` key by recording the time that those keys
are pressed, and checking for a time gap greater than 50ms. This is
necessary to distinguish between the user pressing `Ctrl`+`AltGr`, or
just pressing `AltGr` alone, which triggers a fake `Ctrl` key press at
the same time.

## Validation Steps Performed

I created a test script to automate key presses in the terminal window
for every relevant key, along with every Ctrl/Alt/Shift modifier, and
every relevant mode combination. I then compared the generated input
sequences with XTerm and a DEC VT240 terminal. The idea wasn't to match
either of them exactly, but to make sure the places where we differed
were intentional and reasonable.

This mostly dealt with the US keyboard layout. Comparing international
layouts wasn't really feasible because DEC, Linux, and Windows keyboard
assignments tend to be quite different. However, I've manually tested a
number of different layouts, and tried to make sure that they were all
working in a reasonable manner.

In terms of unit testing, I haven't done much more than patching the
ones that already existed to get them to pass. They're honestly not
great tests, because they aren't generating events in the form that
you'd expect for a genuine key press, and that can significantly affect
the results, but I can't think of an easy way to improve them.

## PR Checklist
- [x] Closes #16506
- [x] Closes #16508
- [x] Closes #16509
- [x] Closes #16510
- [x] Closes #3483
- [x] Closes #11194
- [x] Closes #11700
- [x] Closes #12555
- [x] Closes #13319
- [x] Closes #15367
- [x] Closes #16173
- [x] Tests added/passed
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Area-Input Related to input processing (key presses, mouse, etc.) Resolution-Duplicate There's another issue on the tracker that's pretty much the same thing.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@j4james @leoshusar @zadjii-msft