Add Turkish keymap aliases and sendstring LUT

[fauxpark]

Description

@Ardakilic here you go, please test this :)

Types of Changes

• Core
• Bugfix
• New feature
• Enhancement/optimization
• Keyboard (addition or update)
• Keymap/layout/userspace (addition or update)
• Documentation

Issues Fixed or Closed by This PR

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• I have tested the changes and verified that they work and don't break anything (as well as I can manage).

[Ardakilic]

Hey there, awesome! Thanks @fauxpark

I'll test the keys for keymap_turkish.h this evening, also to test the sendstring_turkish.h, would a macro that types

SEND_STRING("Turkish special chars along with some other strings etc.");

and checking the output suffice? Or are there proper methods which I'm not yet aware of ?

Thanks again for the awesome job!

[ergenekonyigit]

🤘

[drashna]

@Ardakilic That would be the best way to test that out, yeah.

[fauxpark]

Unfortunately the sendstring LUT, by its nature, only addresses ASCII characters. If you need to type Ğ you have to use tap_code(TR_GBRV). Sorry :(

And of course, this will only work if you have the Turkish input layout set in your OS.

[Ardakilic]

Also pinging @bbaserdem here. Although I don't know him personally, he's the first person I stumbled upon while checking Turkish layout for ortholinear keyboards.

[bbaserdem]

Thanks for pinging me, this is nice feature to add. I would also translate to turkish F layout (Instead of the TR_ prefix, maybe use TQ for turkish q prefix? Then we can include a TF prefix for turkish f?)

From what I understand, the sendkeys function only sends keycodes, not unicode sequences. (which would also increase string type time by 5 which is undesirable.) So as @fauxpark stated, the only way to accomplish would be to detect non-ascii characters in string and send them with their specific modifiers. Sendkeys already does this with capital letters and shift, so it should not be too difficult to do. (I would do it cause I am working on my own keymap right now.)

For the record, my implementation was; I wrote some custom keycodes that detected if shift was pressed, and sent the correct case of the turkish letter unicode sequences. But I abandoned that implementation since the unicode is dependent on the us qwerty layout. Correct me if I am wrong. Currently the only thing I do for turkish layout is include some other keys in my planck layout so if i switch to F on software keymap, I have access to all letters. (But planck is too small for all the turkish letters so I seldom do this)

[Ardakilic]

@fauxpark @drashna All I have to do is compile like I do, and these will be included into the compiled firmware, right? Do I have to add anything else to my keyboard files ?

@bbaserdem These keycodes and maps are useful only when you've set Turkish as your OS's language. Aside from that, I do similar approach (E.g: In my handwired 40%, for Ğ and Ü keys, I use raise button + L and ;/Ş respectively which feels right for me), because I'm still using Turkish as input language, and I possibly would to a similar approach if I ever switched to US layout.

[fauxpark]

@Ardakilic you also have to #include "keymap_turkish.h" at the top of your keymap and use the keycodes defined in there instead, but yes, essentially. And to use the sendstring LUT you'd need to just #include "sendstring_turkish.h" as well.

@bbaserdem I don't think it will be that easy to get Unicode sendstring working, due to the sheer number of characters that then must be mapped - probably it will not fit on a 32U4. But if you have an idea, do share it :)

[bbaserdem]

@fauxpark, unicode sendstring method does not work with non-us layouts, at least not on linux. My usecase is I use xkbmap on linux to set my keyboard layout to us-dvorak, and switch between turkish-f and us-q, and have ibus for unicode and emoji. Since my default layout is dvorak, the unicode methods' send string is converted to different letters by libinput, and the hex code is garbled. I have not found any way to ignore xkb keymap in ibus input so I have abandoned using unicode keys. I find that besides these edge cases, I don't find any use in it.

The only non-ascii letters in a turkish string should be ğĞüÜşŞöÖçÇâÂıİ. It should not be hard to come up with exceptions for these 14. Or a better would be to rewrite a copy of the send_keys function that takes in a utf string array as opposed to a char array, and does the same thing as the send_keys function. I find the declaration line here. I never dealt with utf on C before though.

I just checked and TF is not associated with any other language abbreviation. TF_* keycodes can be used to also do a turkish f keyboard database. I do not know how to add suggestions to your pull request, but I can submit it along with my upcoming update to my userspace code.

[fauxpark]

The issue here is that Unicode characters are multiple bytes, and the sendstring LUTs operate only on single bytes. Here's an example SEND_STRING():

And here's what the resulting firmware looks like in a hex editor:

The Ğ character is encoded into the bytes C4 9E: https://en.wikipedia.org/wiki/%C4%9E#Character_encoding
You would need to have a function that detects and transforms these bytes into their Unicode codepoints to send off to the Unicode send function - so this would be more suited for the Unicode feature TBH.

Additionally, my earlier statement that the LUT could be extended was wrong. Ç, the next character in the "extended ASCII" table, is encoded into C3 87... because we're dealing with UTF-8 here. The reason these 128 character tables work is because backward compatibility with ASCII is part of the design of UTF-8.

[bbaserdem]

Checking these strings;

• https://en.wikipedia.org/wiki/%C4%9E#Character_encoding
• https://en.wikipedia.org/wiki/%C3%96#Encoding
• https://en.wikipedia.org/wiki/Dotted_and_dotless_I#In_computing
• https://en.wikipedia.org/wiki/%C3%87#Computer
• https://en.wikipedia.org/wiki/%C3%82#Character_mappings
• https://en.wikipedia.org/wiki/%C3%9C#Computing_codes
• https://en.wikipedia.org/wiki/%C3%96#Encoding

All the first bytes in UTF are C3 or C4. They could be treated as edge cases; since they do not correspond to keycodes as far as I see? Correct me if I am wrong but the send_string_with_delay_p function reads multiple characters for the edge cases. And the send-off does not need to be unicode strings, it can just be keypresses with modifiers activated. (Obviously the implementation would be dependent on the F or Q layouts being used.) I can take a stab at trying it out this weekend. (I think if this request follows through, I can bring my master branch up to date with upstream, and then edit the existing code. I don't think I can submit an amendment to this request since I am not a maintainer, I'm still trying to get the hang of using git and github.)

People would have to specifically work with UTF-8 on their respective code editors, but that should not be a problem.

(My programming skills with C is pretty low; if I am saying something really stupid I thank you for your patience.)

[Ardakilic]

I say it's good to merge. Turkish characters such as Ğ ğ etc. on macros don't work, but for the time being, as mentioned, tap_code is a workaround and works.

[fauxpark]

Added the Turkish F keymap and sendstring LUT.

@bbaserdem I did some Googling and it appears to be fairly trivial to extract codepoints from a stream of UTF-8 bytes, so there is possibly a pathway there to have a sendstring-like function that works with UTF-8 string literals (cf. send_unicode_hex_string()). As I mentioned earlier though, it would have to be part of the Unicode feature, leaving the existing sendstring function as a purely ASCII implementation.

[bbaserdem]

Yes, I agree that it is out of scope with the sendstring functions we have currently. I don't know if there is any merit to code for it; as I certainly would not use it.

I looked at your keymaps, and it is different than my f layout that I was planning on submitting after your fork got pulled in.. I found no dead-keys on the alt-gr, and some symbols have different locations. All the dead keys are on altgr + shift. I am using the Xorg implementation (on arch linux, and using setxkbmap)

% setxkbmap -query
rules:      evdev
model:      pc105
layout:     us,tr,us
variant:    dvorak,f,
options:    grp:caps_toggle

[fauxpark]

This is how it is on Windows:


macOS has yet another set of mappings for the AltGr keys. I don't know whether keyboard layouts on Linux are consistent across all distros, if not I think it may not be worth trying to get them all. For now we only "support" Windows and a handful of layouts for macOS (Italian, French, German mainly).

[Ardakilic]

Awesome, thanks again for all the reviews and the merge 🎉