Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

en input in a ru browser #62

Closed
PEERCRED opened this issue Jun 15, 2021 · 10 comments
Closed

en input in a ru browser #62

PEERCRED opened this issue Jun 15, 2021 · 10 comments
Labels
bug Something isn't working keyboard

Comments

@PEERCRED
Copy link

Here we go again.

Server Dockerfile:

FROM centos:7.6.1810

ENV LANGUAGE=en_US.UTF-8 LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 PYTHONIOENCODING=UTF-8

RUN yum -y install epel-release && rm -rf /var/cache/yum/

RUN curl https://xpra.org/repos/CentOS/xpra.repo >/etc/yum.repos.d/xpra.repo \
 && yum -y install \
    xpra \
    xorg-x11-fonts-misc \
    xterm xev \
 && rm -rf /var/cache/yum/ \
 && sed -i 's/start-new-commands = no/start-new-commands = yes/' /etc/xpra/conf.d/60_server.conf \
 && echo 'bell = no' >/etc/xpra/conf.d/99_nobell.conf

run as docker run --rm --name=xpra_debug --net=host --mount type=tmpfs,destination=/run/user/0/xpra -v/etc/machine-id:/etc/machine-id $(docker build -q .) xpra start --daemon=no --bind-tcp=127.0.0.1:8080 --html=on --start=xterm

On the client, lets use Chrome(ium), and set the first language to Russian (chrome://settings/languages)
image
Connect to the Xpra server (chromium-browser 127.0.0.1:8080), it should say smth like

Handshake complete; enabling connection
 automatic picture encoding enabled, also available:
  jpeg, png, rgb32, rgb24, webp, h264, mpeg1
HTML5 Linux Chrome client version 4.2-r882
setting keyboard layout to 'ru'

When I connect to the server from the current HTML5 client, I see the virtual keyboard, which helps reproducing the issue.
If I click on 1 on the virtual keyboard, I see 1 printed in the xterm. If I click on a letter, say, q, there is nothing printed in xterm, but I see

Utilities.js:40 getFirstBrowserLanguage()= ru
Utilities.js:40 getKeyboardLayout()= ru

in the Chrome devtools JS console in both cases.

Typing cyrillic letters works fine. If the first browser language is English, then typing both en and ru letters seems to work fine.

If you help me with emitting keyboard events from the devtools console, I'll try to provide reproducer for pressing cyrillic letters. I've tried dipatching keydown KeyboardEvents to document.body or document.querySelector("#pasteboard"), but it doesn't seem to work.

@totaam
Copy link
Collaborator

totaam commented Jun 15, 2021

Is q a valid key in the cyrillic ru keymap?
You may want to run your server with -d keyboard to see if it finds a keycode for that key or not. (my guess is that it doesn't)

The solution to this is often to configure the server side virtual keyboard as ru,us and not just ru.

@PEERCRED
Copy link
Author

Is q a valid key in the cyrillic ru keymap?

But my client's keyboard layout is en when I type q. IDK why Xpra sets keyboard layout to browser's language and not to the layout of the key pressed.

You may want to run your server with -d keyboard to see if it finds a keycode for that key or not. (my guess is that it doesn't)

When I press 1:

2021-06-15 16:20:47,778 will try levels: [0, 4, 1, 5, 2, 6, 3, 7]
2021-06-15 16:20:47,779 do_get_keycode(0, '1', True, [], 0)=10 (level=0, shift=False, mode=0, keysyms=['1', 'exclam', '1', 'exclam'])
2021-06-15 16:20:47,779 process_key_action(['key-action', 1, '1', True, (), 0, '1', 0, 0]) server keycode=10, group=0
2021-06-15 16:20:47,780 set_keyboard_layout_group(0) config=KeyboardConfig(ru / None / None), current keyboard group=None
2021-06-15 16:20:47,780 setting XKB layout group 0
2021-06-15 16:20:47,781 filtered_modifiers_set([])=set([])
2021-06-15 16:20:47,782 filtered_modifiers_set([])=set([])
2021-06-15 16:20:47,783 is_modifier(10) not found
2021-06-15 16:20:47,783 handle_key((1, True, '1', 0, 10, [], False, True))
2021-06-15 16:20:47,784 handle keycode pressing    10: key '1'
2021-06-15 16:20:47,784 fake_key(10, True)
2021-06-15 16:20:47,787 modifier_map({})={'control': 4, 'mod1': 8, 'mod2': 16, 'mod3': 32, 'mod4': 64, 'mod5': 128, 'lock': 2, 'num': 0, 'hyper': 0, 'meta': 0, 'shift': 1, 'alt': 0, 'super': 0, 'scroll': 0}
2021-06-15 16:20:47,788 compute_modifier_keynames: keycodes_for_modifier_keynames={'ISO_Level3_Shift': [92], 'Mode_switch': [203], 'Meta_L': [64, 205], 'Control_R': [105], 'Super_R': [134], 'Alt_R': [108], 'Hyper_L': [207], 'Caps_Lock': [66], 'Hyper_R': [253], 'Alt_L': [64, 204], 'Num_Lock': [77], 'Super_L': [133, 206], 'Shift_R': [62], 'Meta_R': [108], 'Control_L': [37], 'Shift_L': [50]}
2021-06-15 16:20:47,789 keys_changed() updated keyboard config=KeyboardConfig(ru / None / None)
2021-06-15 16:20:47,958 process_key_action(['key-action', 1, '1', False, (), 0, '1', 0, 0]) server keycode=10, group=0
2021-06-15 16:20:47,958 set_keyboard_layout_group(0) config=KeyboardConfig(ru / None / None), current keyboard group=0
2021-06-15 16:20:47,959 filtered_modifiers_set([])=set([])
2021-06-15 16:20:47,959 filtered_modifiers_set([])=set([])
2021-06-15 16:20:47,960 is_modifier(10) not found
2021-06-15 16:20:47,960 handle_key((1, False, '1', 0, 10, [], False, True))
2021-06-15 16:20:47,961 handle keycode unpressing  10: key '1'
2021-06-15 16:20:47,961 fake_key(10, False)

When I press q:

2021-06-15 16:21:00,181 will try levels: [0, 4, 1, 5, 2, 6, 3, 7]
2021-06-15 16:21:00,181 do_get_keycode(0, 'q', True, [], 0)=-1, 0 (keyname translation)
2021-06-15 16:21:00,182 process_key_action(['key-action', 1, 'q', True, (), 0, 'q', 0, 0]) server keycode=-1, group=0
2021-06-15 16:21:00,182 set_keyboard_layout_group(0) config=KeyboardConfig(ru / None / None), current keyboard group=0
2021-06-15 16:21:00,183 filtered_modifiers_set([])=set([])
2021-06-15 16:21:00,184 filtered_modifiers_set([])=set([])
2021-06-15 16:21:00,361 process_key_action(['key-action', 1, 'q', False, (), 0, 'q', 0, 0]) server keycode=10, group=0
2021-06-15 16:21:00,362 set_keyboard_layout_group(0) config=KeyboardConfig(ru / None / None), current keyboard group=0
2021-06-15 16:21:00,363 filtered_modifiers_set([])=set([])
2021-06-15 16:21:00,363 filtered_modifiers_set([])=set([])
2021-06-15 16:21:00,364 is_modifier(10) not found
2021-06-15 16:21:00,364 handle_key((1, False, 'q', 0, 10, [], False, True))
2021-06-15 16:21:00,365 handle keycode 10: key q was already unpressed, ignoring

The solution to this is often to configure the server side virtual keyboard as ru,us and not just ru.

I think in the Docker file, I don't configure the server at all. When browser language is English, there seem to be no problem with any of the layouts.

@totaam totaam added bug Something isn't working keyboard labels Jun 15, 2021
@totaam
Copy link
Collaborator

totaam commented Jun 15, 2021

But my client's keyboard layout is en when I type q.

Xpra doesn't know that.

IDK why Xpra sets keyboard layout to browser's language and not to the layout of the key pressed.

Changing the keyboard layout is an expensive operation, definitely not something that can be done fast enough to be done before handling a key event.

When I press q:
2021-06-15 16:21:00,182 process_key_action(['key-action', 1, 'q', True, (), 0, 'q', 0, 0]) server keycode=-1, group=0
2021-06-15 16:21:00,361 process_key_action(['key-action', 1, 'q', False, (), 0, 'q', 0, 0]) server keycode=10, group=0

That looks quite odd. It doesn't find the keycode when pressing (True) but it does when unpressing (False).
The pressed state should not be making any difference to the key mapping strategy.

@basilgello
Copy link
Contributor

basilgello commented Jun 18, 2021

I have the same issue: RU/UA layout works if I select "Russia" or "Ukraine" in HTML5 advanced options and also works if Android on-screen keyboard switches to Russian or Ukrainian, but switching back to English is impossible. I think I will refactor simple-keyboard usage to implement dynamic layout switching and also add full-width layout and floating layout for tablets.

@PEERCRED
Copy link
Author

@basilgello could you test

diff --git a/html5/js/Client.js b/html5/js/Client.js
index f1aca91..368583b 100644
--- a/html5/js/Client.js
+++ b/html5/js/Client.js
@@ -119,8 +119,7 @@ XpraClient.prototype.init_state = function(container) {
 	this.cipher_out_caps = null;
 	// detect locale change:
 	this.browser_language = Utilities.getFirstBrowserLanguage();
-	this.browser_language_change_embargo_time = 0;
-	this.key_layout = null;
+	this.key_layout = this._get_keyboard_layout();
 	this.last_keycode_pressed = 0;
 	// mouse
 	this.last_button_event = [-1, false, -1, -1];
@@ -655,44 +654,19 @@ XpraClient.prototype._check_browser_language = function(key_layout) {
 	 * This function may send the new detected keyboard layout.
 	 * (ignoring the keyboard_layout preference)
 	 */
-	const now = Utilities.monotonicTime();
-	if (now<this.browser_language_change_embargo_time) {
+	if (this.key_layout==key_layout)
 		return;
+
+	this.clog("input language changed from", this.key_layout, "to", key_layout);
+
+	var new_layout = key_layout;
+	if (new_layout==null) {
+		var kb_layout = this._get_keyboard_layout();
+		new_layout = this.key_layout == kb_layout ? "us" : kb_layout;
 	}
-	let new_layout;
-	if (key_layout) {
-		new_layout = key_layout;
-	}
-	else {
-		//we may have used a different layout for a specific key,
-		//and now this new key doesn't need it anymore,
-		//so we may want to switch back to the original layout:
-		const l = Utilities.getFirstBrowserLanguage();
-		if (l && this.browser_language != l) {
-			//if the browser language has changed,
-			//this takes precedence over the configuration
-			this.clog("browser language changed from", this.browser_language, "to", l);
-			this.browser_language = l;
-			new_layout = Utilities.getKeyboardLayout();
-		}
-		else {
-			//this will honour the setting supplied by the user on the connect page
-			//or default to Utilities.getKeyboardLayout()
-			new_layout = this._get_keyboard_layout() || "us";
-		}
-	}
-	if (new_layout!=null && this.key_layout!=new_layout) {
-		this.key_layout = new_layout;
-		this.clog("keyboard layout changed from", this.key_layout, "to", key_layout);
-		this.send(["layout-changed", new_layout, ""]);
-		//changing the language too quickly can cause problems server side,
-		//wait a bit before checking again:
-		this.browser_language_change_embargo_time = now + 1000;
-	}
-	else {
-		//check again after 100ms minimum
-		this.browser_language_change_embargo_time = now + 100;
-	}
+
+	this.key_layout = key_layout;
+	this.send(["layout-changed", new_layout, ""]);
 };
 
 

@basilgello
Copy link
Contributor

basilgello commented Jun 18, 2021 via email

@totaam
Copy link
Collaborator

totaam commented Jun 19, 2021

The changes that this would revert are from #65. (see also #66)
Xpra-org/xpra@9be2370
Xpra-org/xpra@fd21624

As I said before, changing the keymap is expensive and this is something that should be avoided if at all possible. It will block the UI thread for a little while and it will trigger keymap-changed events in all the applications listening for it.

So, I think option 1 is safer, but perhaps option 2 should be used when option 1 fails to locate the keysym?
And to be clear: keyboard handling is best done server side, the client should send all the information it has (adding extra data to the keyboard packets if needed) and let the server choose how to handle this. (ie: MS Windows and MacOS shadow servers won't be using the same functions)

belonesox added a commit to belonesox/xpra-html5 that referenced this issue Dec 5, 2021
@belonesox
Copy link

belonesox commented Dec 5, 2021

I tried patch from #62 (comment) , and I does not work for me.

I am not sure that it same bug about RU/EN layouts, but I reproduced "sticking on EN or RU layout" by connecting from two browsers (after second shared connection, layout switching is broken).

  • First HTML5 connection — All OK, RU/EN OK
  • After second connection "sticking on EN occured".

2min demo of this with this patch applied → https://vimeo.com/653322622
( 3min demo of this without this patch → https://vimeo.com/653319589 )

So, any workaround appreciated… Or should I create another bug?

@totaam
Copy link
Collaborator

totaam commented Dec 5, 2021

I am not sure that it same bug
(..)
after second shared connection, layout switching is broken

Supporting multiple concurrent connections and expecting layout switching to work is not supported.
Please create a separate enhancement request for the xpra server.

@totaam
Copy link
Collaborator

totaam commented May 4, 2022

Perhaps #142 can help with this?

@totaam totaam closed this as completed Aug 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working keyboard
Projects
None yet
Development

No branches or pull requests

4 participants