Implement symbol, alpha, numeric, arrow keyboard event handling

[zeheater]

This pull request attempt to address #4.

It worked without implementing custom soft keyboard by leveraging Android Accessibility API.
Please review and comment.

[bk138]

Interesting approach @zeheater - can you tell us why the PR got closed / what did not work?

[zeheater]

After further testing, i encountered some null pointer exception, and discover some issues in the binding logic where it fail to update the remote buffer when typing using the soft keyboard.

So, i closed the PR. I will open another PR after i fix these issues.

[bk138]

Thanks for the explanation. Can you also elaborate on the pros and cons of this approach versus a 'proper' IME?

[zeheater]

Accessibility approach:

• Pros: This approach do not require user to install a custom keyboard.
• Cons: Google's stance toward Accessibility API used for other purpose than Accessibility is concerning, in the future they may ban / break this approach.

Custom Keyboard approach:

• Pros: Do not need to deal with Accessibility API
• Cons: Required to install and set custom keyboard as the default keyboard.

[bk138]

Accessibility approach:

* Pros: This approach do not require user to install a custom keyboard.

* Cons: Google's stance toward Accessibility API used for other purpose than Accessibility is concerning, in the future they may ban / break this approach.

How does this approach deal with input to arbitrary widgets? Like game surfaces and the like? I think I saw a reference to edittext in the PR...

Custom Keyboard approach:

* Pros: Do not need to deal with Accessibility API

* Cons: Required to install and set custom keyboard as the default keyboard.

Not sure about that. Please have a look at alphavnc, they do the IME approach and there's no custom keyboard to install.

[zeheater]

Accessibility approach:

* Pros: This approach do not require user to install a custom keyboard.

* Cons: Google's stance toward Accessibility API used for other purpose than Accessibility is concerning, in the future they may ban / break this approach.

How does this approach deal with input to arbitrary widgets? Like game surfaces and the like? I think I saw a reference to edittext in the PR...

For my implementation right now, i only care about edittext for basic send keystrokes. That is sufficient for my use case.
But i think i got your point, I haven't thought much about other uses cases such as for gaming. I will try the gaming use case when i got the time.
Also, the timing issue might matter a lot in gaming context. I'm not sure if gaming is in the use case list, i will use this approach.

Custom Keyboard approach:

* Pros: Do not need to deal with Accessibility API

* Cons: Required to install and set custom keyboard as the default keyboard.

Not sure about that. Please have a look at alphavnc, they do the IME approach and there's no custom keyboard to install.

Oh... i didn't know about alphavnc, i will look into it.

[zeheater]

@bk138, i have checked alpha vnc. They do have IME approach
Alpha vnc has a built-in keyboard, in order to provide full soft keyboard support to Android devices prior to version 7.0 (Nougat).
But for newer Android version, they do use the Accessibility API approach. So in that context, it's still required to set the device default keyboard to their soft keyboard for older android api.

[bk138]

I think you can request their source code as they're GPL as well in order to have an in-depth look.

[zeheater]

Btw, i have updated my branch. I think it's a "good enough" implementation for my case. The code quality is still far from ideal.

I still haven't tried it on gaming yet, but it should be no issue as long as it is editable field. There are some issues with some other widget, such as Spinner which i fix by hardcoding "EditText" class name.

You're welcome to check and review. But i think this is it for now.

[bk138]

Thanks, I'll have a look, time permitting. Might take a while though, €€€-work to do.

[zeheater]

No worries... take your time.
Same condition for me too :)

[dkebler]

I have android 12. droidvnc-ng is shown as downloaded app in accessibility menu, but I have no keyboard via my vnc client. (remmina)

Also suggestion is to enable "discussion" for this repo so we can ask support type questions outside of the issue tracker.

[bk138]

@dkebler for question outside the issue tracker, there is https://app.gitter.im/#/room/#droidVNC-NG_community:gitter.im as outline here: https://github.com/bk138/droidVNC-NG#readme :-)