Input Method Kit (IMKit) Apple Radars and Bug Reports

[pkamb]

Opening this issue to collect Radars that have been opened about the Mac OS X Input Method Kit framework.

https://developer.apple.com/documentation/inputmethodkit

Please open bugs on Radar and Open Radar, and post links in this ticket:

https://bugreport.apple.com/

https://openradar.appspot.com/

Input Method Kit tag on Stack Overflow:

https://stackoverflow.com/questions/tagged/input-method-kit

[pkamb]

Input Method Kit reports wrong values for NSEvent charactersIgnoringModifiers in handleEvent:client:
https://openradar.appspot.com/radar?id=21463962
rdar://21463962

---

Input Method Kit cannot catch NSKeyUp events. Chorded keyboards are thus impossible to implement.
https://openradar.appspot.com/radar?id=5041331931971584
rdar://21376535

---

IMKCandidates setCandidateData: method not working in place of IMKInputController candidates: delegate method
https://openradar.appspot.com/radar?id=4960397434028032
rdar://26868067
https://twitter.com/mzp/status/918263309605920769
mzp@0be199e

---

Custom input method app is added to Keyboard Input selector hundreds of times. [NumberInput sample code]
https://openradar.appspot.com/radar?id=5022901556215808
rdar://21376114
https://twitter.com/mzp/status/922610216599830528
image: https://imgur.com/a/mxMCtVA

---

Add documentation that .inputmethod. is required(?) in the bundle identifier of an Input Method Kit input method.
https://openradar.appspot.com/radar?id=5033563544092672
rdar://21376022

[pkamb]

Here are two related Radars for non-IMKit mouse and keyboard handling:

---

Unable to get position of text-insertion caret from AX when no text is selected
https://openradar.appspot.com/radar?id=5026746088816640
rdar://14285519

---

Returning NULL in CGEventCallback function does not kill mouse events
https://openradar.appspot.com/radar?id=5511730692096000
rdar://14123633

[mzp]

Some methods of IMKCandidates don't work well.

• rdar://34944196: -[IMKCandidates candidateStringIdentifier:] always returns 0
• rdar://34911503: -[IMKCandidates showAnnotation:] doesn't work

[mzp]

IMK has some private features(and I want to use it)

• rdar://39295951: Add document: Keyboard Preferences pane
• rdar://39296952: Add API & document: show keyboard layout for input methods
• rdar://39297768: Add TouchBar support to IMK
• rdar://39330175: Add distinguish way English keyboard from non-English keyboard to HIToolbox.
• rdar://39363334: Add document: TISUnifiedUIForInputMethodEnabling

Some properties of Info.plist is unknown where to use:

• rdar://39379605: Mark katakana key of tsInputModeJISKeyboardShortcutKey as deprecated.
• rdar://39380087: Add more document for Info.plist properties of IMK

Other:

• rdar://39297503: Distribute 3rd party input methods via Mac AppStore
• rdar://39380486: Improve Swift binding of InputMethodKit

[pkamb]

• IMKServerInput crashes under ARC when super used for -updateComposition and -cancelComposition. Fixed by overriding methods or perhaps by overriding selectionRange.

• rdar: not written yet

https://github.com/stackia/XIME/blob/master/XIME/Source/InputController.m
pkamb/InputMethodKitBoilerplate@d69a163

[pkamb]

Suggestion from Apple engineer at WWDC for developing Input methods:

1. Move Input Method executable into the Library/Input Methods/ directory via a Run Script build phase on your Xcode project:

    echo "Moving Input Method Kit app to Input Methods directory."
    killall $EXECUTABLE_NAME
    cp -R "$BUILT_PRODUCTS_DIR/$FULL_PRODUCT_NAME" "$HOME/Library/Input Methods"
    #"$HOME/Library/Input Methods/$EXECUTABLE_FOLDER_PATH/$EXECUTABLE_NAME" install;

2. Log in to a fresh OS X user account after installing the input method. Use this account to test the Input Method.

3. Remove the IMKit caches if needed. Other caches are found in the Sandbox container of each Input Method.

    /System/Library/Caches com.apple.IntDataCache.le*

    sudo find/private/vare/folders/ -name "com.apple.IntlDataCache"

[pkamb]

Logging I was made aware of during a WWDC 2016 lab; not quite sure of the intended behavior but perhaps useful to others:

 defaults write -g IMKClientServerXPCTracing 1

[ShikiSuen]

[FB11300759] IMKCandidates (of InputMethodKit) has lack of hardened unit tests.

There are too many bugs of IMKCandidates complained and reported in the recent decade, but it looks like Apple is not dealing with them case-by-case. That's why I feel that sending bug reports one-by-one is not helpful at all in improving the overall experience of IMKCandidates.

https://github.com/pkamb/NumberInput_IMKit_Sample/issues

Conclusion first: All APIs officially exposed in the InputMethodKit shall be hardened by strong and well-designed unit tests in order to make sure that they are fully functional, plus nutrient documentations. Also, I believe that there are new IMKCandidates APIs you want to expose to the 3rd-party developers.

I think that the following things are urgent:

1. All font attributes (incl. font size attributes and langIdentifier attributes) and attachments generated by { InputMethodController.candidates(_ sender: Any!) -> [Any]! } are expected to be reflected in the IMKCandidates panel, and are also expected to be delivered intact as parameters sent to { InputMethodController.candidateSelectionChanged(_ candidateString: NSAttributedString!) } and { InputMethodController.candidateSelected(_ candidateString: NSAttributedString!) }

2. There needs to be a public, swift-friendly API to know the highlighted candidate index of the current candidate panel (or send it as a new parameter to { InputMethodController.candidateSelectionChanged() }. Currently, the identifiers (I dunno whether it is the index) sent back by { selectedCandidate() -> Int } are always NSNotFound.

3. Also allow setting selection keys with characters.

4. There needs to be a new way of interacting with IMKCandidates by not sending NSEvents to it but instead telling it to do specific things (like flipping pages, moving highlight to specific candidate index, expand the horizontal grid list, etc.)

5. Make default candidate keys responsive by exposing { IMKCandidates.perform(Selector(("handleKeyboardEvent:")), with: NSEvent) } to 3rd-party developers, and please document this function well. Currently, IMKCandidates.interpretKeyEvents() is not handling candidate keys at all (so does IMKCandidates.handle(event:)).

6. Please Stop hiding the IMKCandidates beneath NSMenus and Spotlight window by default. At least, I have tried dealing with this by using { IMKCandidates.perform(Selector(("setWindowLevel:")), with: client().windowLevel() + 1000) }, with a miserable result that the candidate runs behind all windows of the current desktop instead: as long as the input method gets process-killed, it happens and you have to reboot macOS to let the IMKCandidates window shown above the client.

Conclusion:

Finally, this report consists of angers and desperations from almost all 3rd-party input method developers for macOS in this decade. Please don't ask me to provide a sample project to expose this. I am not paid by Apple to help you write these unit tests. If you want a sample project, feel free to check vChewing: https://github.com/ShikiSuen/vChewing-macOS (please run make update and compile the target "vChewingInstaller"). Note that this vChewing project is only documentated in Traditional Chinese, regardless that it has UI language support in English and Japanese. You can enable IMK candidate window through the DevZone page in vChewing Preferences window.

P.S.: A powerpoint presentation file written in Traditional Chinese tells how vChewing implemented the IMKCandidates, plus my complaints about its real issues, for your reference: https://docs.google.com/presentation/d/1hYO8U_V4UNQb9PT1K_yzZS_hpzzLCQ_c2GhnoEo8f6s/

[ShikiSuen]

Note: The API setSelectionKeys() is not wrecked but has a hard prerequisite:

Run IMKCandidates().setSelectionKeysKeylayout() first.

Otherwise, setSelectionKeys() won't work.