fix editing with "soft keyboards" (eg. Android, IMEs)

[ianstormtaylor]

Do you want to request a feature or report a bug?

Bug.

What's the current behavior?

The issue with using Slate on Android is complex, and due to a difference in how its OS keyboard is designed. On mobile devices, keyboards are starting to move away from the "keys" concepts in many ways...

• Autocorrect will make changes without any "key" being press.
• Autosuggest will insert words that don't map to keys.
• Swipe-to-type will insert entire words in one go, instead of using keys.
• etc.

Because of this, it sounds like the Android team (reasonably from their point of view) decided to not reliably fire key-related events.

As soft input methods can use multiple and inventive ways of inputting text, there is no guarantee that any key press on a soft keyboard will generate a key event: this is left to the IME's discretion, and in fact sending such events is discouraged. You should never rely on receiving KeyEvents for any key on a soft input method.
—KeyEvent, Android Reference

It sounds like the behavior is:

• The keypress event is never triggered, which is fine for us.
• Text-based keys like a, b, etc. fire a keydown event but always with a key code of 229, indicating that the key is unidentifiable because the keyboard is still busy processing IME input, which may invalidate the actual key pressed.
• Pressing keys like enter fires a keydown event as normal, with an event.key that can be recognized? (This is unclear whether this happens or not.)
• Pressing backspace does not fire a proper keydown event.

A few different resources:

• Great Android postmortem writeup: https://docs.google.com/document/d/1Hex89Di-r-Wfpo1DLAtxpetoX588ziXVoNyC87Je3Xc/edit#
• Chrome "bug": https://bugs.chromium.org/p/chromium/issues/detail?id=118639
• React issues: Key events in Chrome for Android facebook/react#4079 keyboard events do not behave as expected on mobile facebook/react#6176 keyup event not fired on Firefox for Android facebook/react#11231
• VSCode's IME test docs: https://github.com/Microsoft/vscode/wiki/IME-Test
• @danburzo's input methods: https://github.com/danburzo/input-methods

What's the expected behavior?

The fix for this is also complicated. There are a handful of different, overlapping pieces of logic that need to change, to accommodate a handful of different input types...

The first stage is to handle basic insertions, and auto-suggestions...

• Remove the preventDefault in onBeforeInput, so that the DOM is updated, and the onInput logic will trigger, diffing the insertion and then "fixing" it.
• Update the <Leaf> (and other?) components to increment their key such that React properly unmounts and reconciles the DOM, since it has changed out from under it.

This is actually the same starting steps as is required for #2060, so I'd recommend we solve that issue in its entirety first, to work from a solid base.

This fixes the actual text insertion pieces, and probably deletions as well. Splitting blocks can still be handled by enter because it still provide proper key codes.

• Check that all other behaviors that aren't text insertions (eg. splitting blocks) are handled properly without access to many keydown events.

And then there's some selection issues, which apparently can mess up Android's IME (and potentially others) if the selection is manually changed during a composition.

• Prevent re-rendering the editor on compositionstart and compositionend.
• Prevent updating the selection while a composition is taking place.

I think this would solve the 90% case for soft keyboard input.

---

Separately, there's still another question of how to properly handle these kinds of behaviors other plugins. For example if a plugin uses backspace at the start of a block to reset that block, that won't work on Android. So after solving the input issues, we need to step back to an architectural level and solve this plugin handling problem. But that can wait.

[ianstormtaylor]

I've done some input tests in different browsers and devices using Dan Burzo's input tester.

I think the only way we're going to arrive at proper Android support is to use a tool like this, and take extremely detailed recordings of inputing the exact same text across the different platforms. Otherwise the intricacies of the ordering of events, and when compositions do or do not start is going to be too complex to guess.

I've started with a few, we'll need more. I think we'll need:

• Inserting text.
• Inserting IME text.
• Inserting a word via auto-suggest.
• Inserting a word via gesture typing.
• Auto-correcting a word.
• Deleting characters backwards.
• Deleting characters forwards.
• Delete characters backwards, across word boundaries.
• Selecting a word, then deleting it.
• Selecting a range across blocks, then deleting it.
• Moving to the end of a word.
• Moving to the middle of a word.
• Moving to the middle of a word, then inserting text.
• Splitting a block at the end.
• Splitting a block in the middle.
• Joining a block from the start.

Even from just the ones I did, we can already see complex composition behaviors on Android that we're going to have to solve for:

• Moving the cursor into or to the edge of a word, starts a new composition immediately (even without any input) with the word as content.

• Deleting characters backward doesn't trigger any useful key events, but it does update the composition ticking down the characters in a word.

• All text insertions on Android is via composition events, so in solving for its composition handling, we'll probably get desktop IME support in the process.

I think we'll need to store additional information in these tests of what the current window.getSelection() range is, because that is key to understanding how much information we can get from these composition events. Someone might need to build a custom input tester for Slate specifically to make this easier.

[thesunny]

@ianstormtaylor

You are making a lot of progress here. I’ve committed next week to working on mobile support but by a large margin, you are more qualified to solve this than I.

I think we might make more progress if you give me any menial tasks, research, etc. In order to help you. Don’t feel obligated to give me work but if you have any tasks, I will work on them full time for you starting Monday. Let me know.

[ianstormtaylor]

@thesunny thanks! One of the most helpful things would be getting more of those detailed event samplings across devices/browsers for each type of edit. That would allow anyone who works on this in the future to reference them to make sure they're thinking through the event logic correctly.

Without those I think we'd just be fumbling around guessing what logic we need to add to the After plugin to get composition events working.

[thesunny]

Okay, I will start with that Monday

[Nantris]

@ianstormtaylor Dan Burzo's input tester is exactly what I was looking for the other day to send you. Glad you found it/already had it.

@thesunny and @ianstormtaylor how/where are you laying down code for this? I don't think there's much I can contribute in the way of code in the short term, but I'm happy to test ideas on my Android device and help any other way that you two need.

[thesunny]

I created DropBox Paper documents. I set up a testing environment and put together all the tests below

So, a few differences from @ianstormtaylor is that I tested 6 different versions of Android and they all provided different values. There are more differences within the different Android versions than across all browser/OS versions. iOS, Safari and Chrome are very similar.

• Inserting text.
• Inserting IME text
• Inserting a word via auto-suggest.
• Inserting a word via gesture typing
• Auto-correcting a word
• Deleting characters backwards
• Deleting characters forwards
• Delete characters backwards, across word boundaries
• Selecting a word, then deleting it.
• Selecting a range across blocks, then deleting it. (1)
• Moving to the end of a word, from inside the word
• Moving to the start of a word, from inside the word
• Moving to the middle of a word, from the left
• Moving to the middle of a word, from the left, then inserting text
• Splitting a block at the end. (1)
• Splitting a block in the middle. (1)
• Joining a block from the start. (1)

(1) Forked the @danburzo app to accept HTML tags like <p></p> and can be found here

I'll keep updating this list.

[Nantris]

@thesunny can you list the browser versions for the Android tests? I'm guessing Chrome 68?

Also, we should agree on which field we're using in that input tester, either the Simple DOM contenteditable or the React-managed contenteditable.

I think it's reasonable to expect Slate to only have out of the box mobile support via React. This would simplify the task, I think. What do you think @ianstormtaylor ?

I believe the only events we should be relying on, if we want the widest base of support on Android, are:

• compositionStart
• compositionUpdate
• compositionEnd
• deleteContentBackward (maybe)

FYI when typing, each keystroke updates the composition, but choosing a suggestion ends the composition and creates the final word like this:

compositionData: Barnac
Suggestion: Barnacles

If you click Barnacles, the final word is constructed like this.

compositionEnd Barnac
insertText les

Barnacles

after this word, Google suggests the word, "the" - If you click this word, it is inserted like this:

insertText the (there is a space inserted before the word)

Note, the above data is from the React contentEditable, but appears to be the same in the simple contentEditable.

[thesunny]

The browser version I believe is locked to the Android API version. I'm using whatever Chrome came with the specific Android version. I guess one thing where you can help is to look up which Chrome comes with which Android version.

I'm using the simple DOM contenteditable because that's what Ian used in his examples.

I was considering writing a web App that would store all this information in a database so I wouldn't have to screenshot everything and we'd also have the raw data as JSON somewhere. I'm not sure how long that would take though so for now I'm just manually see if I can get this all done.

I'm surprised how big the differences are between each Android version.

[thesunny]

@ianstormtaylor Just a heads up that I'm going to redo your Input Tests adding the multiple Android versions.

Edit: Completed tests that don't require blocks as the Input Tester doesn't support them.

Edit: I'm going to go back and add tests for Android API 27. It's a minor upgrade Android 8.0 - 8.1 but just to be safe.

Edit: Also missed Android API 24 (which was mistakenly API 23). Fixing with an @v1.1 to identify that it's been fixed.

Edit: Forked input-methods so that the contenteditable contained two <p> blocks with some text in it in order to complete the tests. https://thesunny.github.io/input-methods/index.html

[ianstormtaylor]

@thesunny these are absolutely amazing! Thank you!

Having them for multiple versions of Android is a great idea. And using the non-React input is best too, since we need to know what exact DOM events are fired, regardless of how React chooses to interpret them. Because we'll probably need to bypass React's event handling logic in places, until they catch up.

Another question for us to answer would be what are the usage levels of the different Android versions.

Regardless, starting by trying to get things working with the latest versions that have Chrome's that fire beforeinput seems best, since that makes things easier I think. There's a decent change that adding support for some of the older ones would be much harder.

[thesunny]

@ianstormtaylor popularity of platform can be found here but unfortunately beforeinput support starts at API27 which has only 2% market share.

If we go back to API 23, we can capture 66.4%. The biggest version in distribution is API 23 which has 23.5% distribution.

I'm just looking through some of the charts and looking for some patterns.

In Inserting Text I think we may be able to get away with:

• Ignore keydown events. This is because a keydown will happen before a compositionstart so we can't know if a keydown is part of a composition until later. We can't know if we want to process based on the keydown event.
• We process all keyup events that don't happen within a compositionstart and compositionend using a DOM diff. In the test, this is always a space (presumably because it doesn't require composition since it's valid in itself).
• When there is a compositionstart then we ignore all the keystrokes. When a compositionend event happens, then we do a DOM diff to create a Slate insert operation.

Some analysis I did on other input methods to see if this strategy holds:

• Japanese IME: It appears valid for IME in API 28 as well.
• Gesture typing: appears valid for gesture typing
• Moving to the middle of a word then inserting text. This appears to work but in some API versions, the text is wrapped in a composition and in others they are just simple key events with no composition. Either way, the logic still works.

It seems like the code might be written something like:

• For Android only, we allow all regular insertion keypresses to all flow through naturally (not including ENTER, BACKSPACE, etc)
• If we find a compositionstart event, we block all operations until a compositionend event materializes. At the time o f a compositionend, we do a diff of the DOM and create an insert op with {isNative: true}.
• If we find a keyup event that is not wrapped in a composition we also do a diff and create an insert op with {isNative: true}

If we aren't in Android, we could theoretically do a diff and use the same code base for everything; however, this might be a performance issue. On the other hand, it could potentially help performance if we debounce key events and end up with a bigger diff and one insert operation at time of keyboard rest. I think we should think about this.

Of course we'd still have to handle ENTER, BACKSPACE, etc. individually but we can probably make a short list of operator keys which is probably ENTER, BACKSPACE, SPACE, and PUNCTUATION.

Usually we don't have Plugin actions from typing in Slate unless they are accompanied with one of these keys anyways. For example, a markdown plugin might use * for a bullet or #heading but all of these are punctuation. I also use o for an empty task myself but that has a space in it. So we would have to limit special Plugin handling to be around these keys but that might be acceptable and it's also a good proxy for the special handling to work in Android anyways because Android will always composition normal words so that key handling would fail in Android.

The plugins may need a different API though that exposes something like onInsertText and onNonWordKey. In this API \we can't do something like have the input onetwothree be converted on the fly to 123 as we type it but we could have one two three work because of the spaces calling an onNonWordKey. This may be a feature since onetwothree live converting could never work in Android due to the way it composes.

[ianstormtaylor]

@thesunny great points all around.

It sounds like one thing we need is a list of key presses that don't trigger compositions and instead use keydown to insert directly? It sounds like space is one of these? But we need a full list. (And as you mention, plugins that want to support Android would need to restrict their keydown usage to these keys, or we need to come up with some new abstraction later.)

Do we need to worry about isNative: true for now? Or can we re-render once the composition has finished with compositionend? Being able to punt that (even just for getting it working with later Android versions) would make it a lot easier.

Fair enough for backwards compatibility. I think whoever implements it would likely want to start with API 28, since it's the most standardized. Or even just ignore Android quirks to start and focus purely on the compositionstart/update/end trifecta. And then handle all of the specific quirks around compositions that need to be ignored.

[thesunny]

@ianstormtaylor

I'll do some more research on what keys have special cases with respect to composition like space. I assume enter is one but I'm not sure about punctuation.

That's an interesting idea about not worrying about isNative: true. So I guess the idea would be that we simply would not update the state (and hence not trigger a re-rerender) until compositionend? Feels like this could work...

In my opinion, we should focus on the compositionstart/end events. Everything I've read on this issue across all of these Editors suggests to me that it is impossible to use the events to reconstruct Editor state without doing a DOM diff with reasonable coverage of Android versions. Since we have to go there eventually, why not start with it. I've designed an algorithm for the DOM diff which I think (hope) will be quite simple to implement so it may not be that hard to get this working.

My next goal is to see if I can create a sample contenteditable React-based editor that uses compositionstart, compositionend and DOM diffing to sync a Slate like Editor State. If I can get that working, we can see how that can merge into Slate proper.

Edit: A preliminary check just on Android 28 suggests that punctuation doesn't trigger composition.

[thesunny]

@ianstormtaylor @slapbox

OMG! I'm super excited!

I created a working prototype. It's a simple React contenteditable editor (one text node in a div) and tested this on three Android versions (including the newest and oldest API versions) and so far it works perfectly!

• composition works
• IME works
• gesture writing works
• auto correct works
• suggestions work

It's generalized enough that it works on desktop with no code changes.

I created a custom editor because I don't know Slate internals well enough yet but I tried to make it as close to Slate as possible.

• It uses insert_text and remove_text ops with the same semantics as Slate (but without path and marks to keep prototype simple)
• They are applied to a text state (similar to Slate's editor state but simplified to one text node).

It works like this:

• It reconciles on keyup unless the keyup is within compositionstart and compositionend events.
• It reconciles on compositionend events
• It uses text diffing to generate the operations and applies them to Editor state

I also track selection and restore it after update but Slate already does that so can be ignored.

Here's a screenshot for now...

I'll try and get this published tomorrow to get some feedback. There will probably be issues but it's promising so far!

[thesunny]

@ianstormtaylor

Here is where you can access the Live Composition Editor

This is the GitHub Repo for Composition Editor

If you type in it from most browsers, it is diffing on every keystroke to create the insert operations and then the Editor is being rendered. After render, I set the selection so the cursor doesn't get lost.

If you use it from Android, when you start a composition via the compositionstart event which happens for most input, the Editor waits until compositionend and then does the diff and creates the operation. You can see this in the operations dump as instead of a single character insertion, you'll see an insert_text with multiple characters in it. In some cases you will also see a remove_text.

I dump the Editor state for transparency which includes all the ops or operations that have been applied.

You can reset the editor to the default text and empty the ops by clicking the Reset button on the page.

@slapbox can you test this on your Android device?

[thesunny]

@ianstormtaylor

I think to implement this, we would have to change the Plugin event handler properties.

Here's my recommendations.

• Add an onCompose synthetic event handler property on plugins.
• deprecate onKeyDown and onKeyUp because it will be unpredictable in Android and could break it

We could have onCompose be the default handler for insertion of characters even for desktop where they'd always compose 1 character at a time. We could consider calling it something like onInsertText but the issue is that in some cases (like switching a word using a suggestion) we'd actually have a remove_text operation along with the insert_text operation. Maybe onChangeText? But onCompose feels okay to me even if it's a little misleading.

My suggestion is to replace the key handlers with a whitelist of specially named key handlers for all the cases we want to make sure Slate can handle. This is safer than having a catch all like keydown and assuming the user will know, for example, that letters in a composition won't fire the event or that they need to not preventDefault on those events or that they shouldn't modify Slate during a composition.

Here's handlers I'd recommend to replace onKeyDown.

• onHotKeyDown which would fire whenever a key combination (i.e. a character in combination with CTRL, ALT, OPT or CMD is pressed) like CMD+B or CMD+SHIFT+2.
• onSpace
• onEnter
• onDeleteBackward
• onDeleteForward
• Or possible combine the two above into onDelete with an event property that specifies the direction? Not sure but I kind of like them separate so as to avoid confusion...

I will look into punctuation but I'm not hopeful that we can rely on these being 100% handled outside compositions because of contractions like can't and isn't.

[ianstormtaylor]

@thesunny I think it might be best to separate the work of "getting Android working" from "getting the plugin system fully adapted for compositions". Because the latter is going to be more work, and require thinking lots of different API considerations through.

---

As for how events flow through the editor...

1. The <Content> component is the one that renders the contenteditable element in the DOM, and attaches events handlers when it does.
2. For handlers where we need the native DOM semantics, instead of React's synthetic event semantics, we have to attach them in componentDidMount to the DOM node.
3. The <Content> component itself is rendered in the After plugin's renderEditor hook, and it passes in the <Editor>'s handler callbacks.
4. The editor takes any event and runs it through the stack which cascades through all of the plugins.

That's the current flow. There's a bit of confusing indirection around how the <Content> element is rendered. But for the most part you can just look directly from <Content> to plugins and ignore it. In the future we may be able to simplify that stuff.

---

As for how to handle compositions in plugins. I don't think we need to deprecate onKeyDown/Up yet, because Android helpfully fires the keys with 229 codes so that they won't ever be recognized, last I checked. There may be cases where this doesn't work, but those should be researched/documented well before we make a decision.

I'd rather avoid having onSpace, onEnter, etc. all be their own handlers. As for onDeleteBackward and onDeleteForward, see the "commands" concept discussion.

[thesunny]

@ianstormtaylor Thanks for the feedback.

Just working my way through everything now.

[thesunny]

Here is my execution plan for now

• disable state updates in Slate from relevant events
• allow native updates to flow through to contenteditable div
• generate diff by locating relevant slate node and comparing it to DOM node (attach initially to keyup events)
• use diff to create operations to modify state and rerender
• now do it with composition events
• enable back all the event handlers and such that are necessary to make slate work properly like hitting enter key

Testing
For all these tests we need to make sure (a) the changes are in the Editor State (b) the selection is in the right spot

IME

• go to end of a word and type hideki in Japanese
• go to start of a word and type hideki in Japanese
• go to middle of a word and type hideki in Japanese

Gesture Writing

• cursor to end of word then use gesture writing
• cursor to start of word then use gesture writing
• cursor to middle of word then use gesture writing

Regular Writing

• enter a full word then press space
• enter a full word then press enter
• enter 2 letters then select a suggestion
• type "hi" at end of line then hit "enter" (broken in API28)

Suggestions

• click on a word and select an alternate suggestion
• click on a word, type a letter, select an alternate suggestion

Auto-correct

• type cant and let it auto-correct to can't
• type can and cursor away. Move cursor back to the end of can and type t and then let auto-correct fix it to can't

Physical Keyboard

• type "Hello World" pretty quickly. Make sure there are no duplicate characters or incorrectly positioned selection.
• hold the backspace key down

[Nantris]

@thesunny is there anything in particular I can be of help testing? I checked out what you've put together and it looks very useful. All input and output is rendered to the editor as expected.

One note, not a problem of any sort; I'm surprised by how backspacing is handled. Backspacing away the last character of a completed word fires a single remove_text event, and then another one doesn't fire until you have removed the entire word. Compositions are strange.

[birtles]

Thanks for working on this. Please do test with Firefox on Android too. We are looking to update Firefox's behavior (see discussion in w3c/uievents#202) and would appreciate your feedback.

[thesunny]

@birtles You're welcome. I'll test out Firefox once I get Android Chrome working.

As a quick update, I have many things working properly now although I'm not using the diffing algorithm yet and just the replace algorithm that currently exists. It's pretty much working except there are issues with selection placement, the fact that Android doesn't always have an onCompositionEnd event (which makes no sense) and also the composition events are fired before the content is actually in the DOM.

I will add the special cases we need to test for in the execution plan above.

[Nantris]

@thesunny can you give an example of a situation where onCompositionEnd doesn't fire?

[robbertbrak]

@thesunny: Great to hear that using MutationObserver is working out so well! 👍 With Draft (which now uses MutationObserver for all composition-based input, not just on Android) I have also seen good results.

Just one question though: when you say that "everything works" on Android 8.1 / 9, what exactly do you mean? In my experience, whether composition-based input works on Android is not just limited to the API version, but also varies depending on:

• Keyboard (Gboard, Huawei Swype, Google Voice Typing, Samsung keyboard, ...)
• Language (Chinese, Korean, Japanese, Arabic, ...)
• Browser (Chrome, Firefox, ...)
• Keyboard variant and settings (Correct words while typing yes/no, Pinyin / Handwriting, ...)

[thesunny]

To clarify I only meant that all my manual tests pass on Android Chrome on Gboard with the exception of continuous backspace. I’m sure there are more bugs which is why I’m asking for bug reports.

My tests do include gesture writing, foreign IME but not some of the others; however if they work as compositions then they theoretically should work. I did nothing extra for IME or gestures for example but they both work.

The initial goal is to get the most common use case for Android at an MVP (mostly works and no unrecoverable crashes) then build on there.

I could see a switch to using only MutationObserver being a path for Slate and I even had Desktop Chrome working on MutationObserver while developing the Android version; however that’s probably a decision ultimately for Ian.

[thesunny]

Made the PR for Android support using mutations here

#2853

[Macil]

I've reported two bugs affecting Android Firefox and Chrome: #2951 and #2952. I figure they're related to this issue but that it might be good to make issues for each of them.

[ianstormtaylor]

I believe that this may be fixed by #3093, which has changed a lot of the logic in Slate and slate-react especially. I'm going to close this out, but as always, feel free to open a new issue if it persists for you. Thanks for understanding.

[fainir]

Hey, where can I find code example for this demo https://thesunny.github.io/slate/#/composition/split-join or any other example that works well on Android? This is the only demo I found that works fine on my Android device

[Nantris]

@fainir likely somewhere in here: https://github.com/thesunny/slate/

Sorry I can't be more helpful with an exact location. 0.47.x doesn't work perfectly with Android, but works much better than previous versions. The biggest issues I have using Google's GBoard is holding down backspace leads to everything getting mangled.

[fainir]

@fainir likely somewhere in here: https://github.com/thesunny/slate/

Sorry I can't be more helpful with an exact location. 0.47.x doesn't work perfectly with Android, but works much better than previous versions. The biggest issues I have using Google's GBoard is holding down backspace leads to everything getting mangled.

Tested on my android device and and it has the same bugs as this demo https://www.slatejs.org/examples/richtext. The only demo I found that works well on my device is https://thesunny.github.io/slate/#/composition/split-join that mentioned few comments above by @thesunny. Does anyone has the code for this demo? Why isn't working on the main demo? It is a huge bug and @thesunny fixed it so I think it should be implemented inside the package and the main demo.

[Nantris]

@fainir you can find out more about that situation here #3573

The code is in that repo, but may not be at the tip of the master branch. It looks like the master is on 0.50.x already.

Basically @thesunny did a ton of work to get it working on 0.47.x, but the project is still in beta and the API is often changing. 0.50.x was a total rewrite by @ianstormtaylor to address a lot of other issues, but making those changes wasn't done in coordination with @thesunny (it would have been basically impossible on a volunteer basis.) If you need Android support stick with 0.47.x. I'm not sure about a demo for that version, but the docs for it are still available alongside the latest docs.

[fainir]

Thank you very much. I downgraded to 0.47.9 and it works :)
I suggest that project's owners would add notice to the readme file that version 0.5.x is not supported for Android and show demo of 0.47.x with Android support because I assume that Android support is very critical for many users.
Thank you very much! great project :)

[vasanthps]

@slapbox Thank you for this amazing project. Is there a roadmap to add support for android devices in 0.5.x? Thanks!

[Nantris]

@vasanthps just to clarify @ianstormtaylor is the amazing person who leads the Slate project.

There's no official roadmap to add support for Android devices and it's not planned right now, hence #3573.

[baljeet-aulakh]

@thesunny @ianstormtaylor I am using Slate v47 and it's working perfectly on desktop and iOS devices but onChange is not triggering on Android devices. I would greatly appreciate any suggestion. Thanks.

[thesunny]

Hi @baljeet-aulakh

Android uses mutations and not the onChange event. If onChange was wired to be called from a mutation, it would not behave in the same way because you could not preventDefault to stop the event.

[baljeet-aulakh]

Hi @baljeet-aulakh

Android uses mutations and not the onChange event. If onChange was wired to be called from a mutation, it would not behave in the same way because you could not preventDefault to stop the event.

Thanks @thesunny
Is there any documentation that can help me get Slate 47 to work on Android phones?
It seems like I will have to configure slate differently for Android cuz it's working fine for desktops and iOS phones.

[thesunny]

@baljeet-aulakh

Sorry, I may have been confusing your query. For clarity, the onChange should fire to update the Slate DOM but the onChange event won't work properly on the contentEditable. Honestly, it's been a while since I looked into the internals so you may have to take my thoughts with a grain of salt!

[clauderic]

@thesunny I'm running into the same issue with Slate 0.47.9.

Tested on a couple of different Android versions using BrowserStack (Android 10, 9, 8) and as far as I can tell the editor.value is rarely updated, and the <Editor onChange={} /> prop is only ever called when deleting characters with an expanded selection.

[thesunny]

It's been a while but this may be due to how Android decides when a composition ends which is when changes are committed.

It is possible for a composition to span multiple locations. Android is the only browser that does this.

Selection deletes always cause a change to happen I believe. But deleting/backspacing within an element won't. But backspacing across a boundary (ie. between two spans) will.

The reasons for this have to do with a balance between performance and consistency. If we reconcile too often, we can have consistency issues because we are trying to reconcile while android is still executing mutations. We have an escape hatch where when things get so far out of sync that React crashes, that we have an error boundary that just rebuilds everything from scratch. But that's jarring so we don't want to see that often. It's better than a fatal crash though.

I'm not working on that code anymore but if you want to take a shot at fixing it, a possible solution or improvement could be to identify when there is a change in selection even if it's in the middle of a composition and then reconcile the browser DOM with the Slate DOM.

If you do take a crack at it, make sure to test against all the test cases in the demo that I wrote specifically for compositions or else you may end up putting the editor in a worse state than it is now.

[clauderic]

a possible solution or improvement could be to identify when there is a change in selection even if it's in the middle of a composition and then reconcile the browser DOM with the Slate DOM.

Hey @thesunny, I'd be happy to look into that if that was the only thing that was broken, but unless I'm missing something it seems like the Slate value is never updated on Android, and therefore the onChange callback is never called.

As mentioned above, the only working scenario I have seen is with an expanded selection and pressing backspace, this will indeed trigger an onChange, otherwise nothing seems to work.

I'm trying to see if this could be a regression at some point before 0.47.9, do you remember this working properly at some point?

[clauderic]

I went as far back as this commit locally and I still was seeing the same behaviour 7d4062c

Maybe I'm missing something?

[thesunny]

It was working at some point so it may be a regression but it's also possibly a change in Android which may be causing the issue. I haven't looked at it since about a year ago so I don't really know what the state is. I'm pretty certain it was working at some point though as some people were relying on it. Also, a lot of the timing issues that I had to deal with had to do with how React re-renders during editing so those issues wouldn't have popped up if onChange never got called.

Also try clicking outside the editor (ie. lose focus) to see if at least that calls onChange. I believe that forces the composition end. If that's not working, it feels like it's a regression or an Android change that is causing it not to work anymore.

[Jtfinlay]

Is there an open issue tracking this I can monitor? Latest version is not playing well with GBoard

[Nantris]

@Jtfinlay it's not planned per se, but there's some hope.

See #3786 and #3573

[jesusgp22]

I still see this issue with

"slate": "^0.58.4",
"slate-history": "^0.58.4",
"slate-react": "^0.58.4"

Selecting all the text and deleting it triggers a crash

[ShanonJackson]

Never before has there been such a team! Nice job @thesunny @ianstormtaylor.

I don't actually use the library but stumbled across this while trying to support Android/IoS for my own ContentEditable stuff and it's been a massive help.

[TrySpace]

I'll link this issue: udecode/plate#1347 because it seems to be related

And what about https://github.com/danburzo/input-methods?