Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some input method fixes (take 2) #2113

Merged
merged 7 commits into from
Feb 27, 2024
Merged

Some input method fixes (take 2) #2113

merged 7 commits into from
Feb 27, 2024

Conversation

lilydjwg
Copy link
Contributor

This is continuation for #2111.

This is a draft as I haven't tested it for long. Also, it needs a fcitx5 patch (to not send superfluous release events).

@ammen99
Copy link
Member

ammen99 commented Jan 23, 2024

Continuing our discussion from the previous PR, I was thinking of using the serials like this:

diff --git a/src/core/seat/input-method-relay.cpp b/src/core/seat/input-method-relay.cpp
index 9689ba9d..e7386534 100644
--- a/src/core/seat/input-method-relay.cpp
+++ b/src/core/seat/input-method-relay.cpp
@@ -50,30 +50,16 @@ wf::input_method_relay::input_method_relay()
         auto evt_input_method = static_cast<wlr_input_method_v2*>(data);
         assert(evt_input_method == input_method);
 
-        // FIXME: workaround focus change while preediting
+        // When we switch focus, we send a done event to the IM.
+        // The IM may need time to process further events and may send additional commits after switching
+        // focus, which belong to the old text input.
         //
-        // With input method v2, we have no way to notify the input method that
-        // input focus has changed. The input method maintains its state, and
-        // will bring it to the new window, i.e. a half-finished preedit string
-        // from the old window will be brought to the new one. This is undesired.
-        //
-        // We ignore such commit requests so it doesn't have any affect on the
-        // new window. Even when the previous window isn't preediting when
-        // switching focus, it doesn't have any bad effect to the new window anyway.
-        //
-        // Preediting can be disabled (selectively or globally) because
-        // application bugs were observed. In this case, we see only commits
-        // but no preedit strings, so we need care the timing.
-        static std::chrono::milliseconds focus_change_duration{100};
-        if (last_focus_changed.has_value())
+        // To prevent this, we simply ignore commits which do not acknowledge the newest done event from the
+        // compositor.
+        if (input_method->current_serial < last_done_serial.value_or(0))
         {
-            auto elapsed = std::chrono::steady_clock::now() - *last_focus_changed;
-            last_focus_changed.reset();
-            if (elapsed < focus_change_duration)
-            {
-                LOGI("focus just changed, ignore input method commit");
-                return;
-            }
+            LOGD("focus just changed, ignore input method commit");
+            return;
         }
 
         auto *text_input = find_focused_text_input();
@@ -173,7 +159,13 @@ void wf::input_method_relay::send_im_state(wlr_text_input_v3 *input)
     wlr_input_method_v2_send_content_type(input_method,
         input->current.content_type.hint,
         input->current.content_type.purpose);
-    wlr_input_method_v2_send_done(input_method);
+    send_im_done();
+}
+
+void wf::input_method_relay::send_im_done()
+{
+    last_done_serial = next_done_serial;
+    next_done_serial++;
 }
 
 void wf::input_method_relay::disable_text_input(wlr_text_input_v3 *input)
diff --git a/src/core/seat/input-method-relay.hpp b/src/core/seat/input-method-relay.hpp
index 036b730d..eca8cf55 100644
--- a/src/core/seat/input-method-relay.hpp
+++ b/src/core/seat/input-method-relay.hpp
@@ -8,7 +8,6 @@
 
 #include <vector>
 #include <memory>
-#include <chrono>
 
 namespace wf
 {
@@ -23,7 +22,11 @@ class input_method_relay
         on_input_method_new, on_input_method_commit, on_input_method_destroy,
         on_grab_keyboard, on_grab_keyboard_destroy, on_new_popup_surface;
     wlr_input_method_keyboard_grab_v2 *keyboard_grab = nullptr;
-    std::optional<std::chrono::time_point<std::chrono::steady_clock>> last_focus_changed;
+
+    std::optional<uint32_t> last_done_serial;
+    uint32_t next_done_serial = 0;
+    void send_im_done();
+
     text_input *find_focusable_text_input();
     void set_focus(wlr_surface*);
 
@@ -37,8 +40,6 @@ class input_method_relay
         {
             set_focus(nullptr);
         }
-
-        last_focus_changed = std::chrono::steady_clock::now();
     };
 
     bool should_grab(wlr_keyboard*);

Does this not work to prevent the old state from interfering with the new focus when changing focus?

…perfluous release events

These events may be sent by the IM, and meant to reach the client.

check:

* in firefox, no text input, press ctrl-l, there should not be
  continuous l being inserted into the urlbar
* in firefox, with text input, press super+some key to switch workspace
  and back (but don't release super inbetween). click in firefox, super
  should not be consider pressed
* with text input, does the switcher plugin activate and deactivate correctly
IM sends release key events when deactivated, which caused e.g. the
switcher plugin to deactivate early. The plugin has a higher priority
when activating, but lower for subsequent key events.
fcitx5 no longer commit_string on deactivating.
@lilydjwg
Copy link
Contributor Author

We can get rid of the workaround now. fcitx5 no longer commits that string.

@ammen99
Copy link
Member

ammen99 commented Jan 24, 2024

We can get rid of the workaround now. fcitx5 no longer commits that string.

I think we still should do the thing I linked with the serial check. Namely, imagine the following scenario:

  • A is focused
  • Fcitx5 sends some commit for A
  • Compositor switches focus before receiving the commit (might happen because wayland communication is async)

Of course this is a pretty rare race condition but it still might happen. So it would be great if you could test it and see whether anything breaks, if not, I think it should stay in that reduced form.

@lilydjwg
Copy link
Contributor Author

Tried with the patch. It didn't work. I mean, it does no good nor bad (I used a previous commit of fcitx5 to test the issue).

Here's the debug log from fcitx5, grepping zwp_input_method_v2:
fcitx5.log

(In the log, I only typed wq without committing and switched focus. The focus just changed didn't appear. It did appear in some but not all later tests.)

Btw, there was a missing wlr_input_method_v2_send_done call in send_im_done.

@ammen99
Copy link
Member

ammen99 commented Jan 24, 2024

Tried with the patch. It didn't work. I mean, it does no good nor bad (I used a previous commit of fcitx5 to test the issue).

Ok yeah .. with the new investigation you did on fcitx5 it turns out that this workaround couldn't have fixed the earlier problem because they were sending the old state even after deactivation. If the workaround (with your fix, thanks for it) doesn't harm anything, I'd like to include it as well simply because it might prevent a rare race-condition (which is apparently different from the bug you were experiencing before).

Also I see that you still do the thing with grabs, is this still really needed when fcitx5 doesn't send fake release events?

@lilydjwg
Copy link
Contributor Author

lilydjwg commented Jan 24, 2024

I'd like to include it as well.

Yes I can include it, as long as I don't find issues in the following days.

Also I see that you still do the thing with grabs, is this still really needed when fcitx5 doesn't send fake release events?

It wouldn't need but fcitx5 does send them. From the developer of fcitx5, sway seems to no longer need these release events but hyprland definitely needs them. Also I got two times that an Enter key is thought to be pressed by xwayland applications but couldn't reproduce it more.

@lilydjwg
Copy link
Contributor Author

It wouldn't need but fcitx5 does send them.

Maybe not. There was an issue that a backspace delete to the start in DDNet (a sdl2 game). I didn't investigate before it was gone due to some changes I'm not sure.

Or we'll alternate between two keyboards, sending a lot of keymap and
repeat_info events to all clients.
@ammen99
Copy link
Member

ammen99 commented Jan 29, 2024

@lilydjwg I see that fcitx5 got a new commit which disables release events for non-repeat keys like the alt modifier. So probably we don't have to differentiate between grabs and non-grabs anymore.

@lilydjwg
Copy link
Contributor Author

Thanks for letting me know. I'll try a few days without that.

@lilydjwg
Copy link
Contributor Author

switcher still behaves unexpected: tab is released by the IM, so alt-tab switches to the next window without the user release the key.

@ammen99
Copy link
Member

ammen99 commented Jan 29, 2024

switcher still behaves unexpected: tab is released by the IM, so alt-tab switches to the next window without the user release the key.

Tab should never be sent to the IM in the first place.

@ammen99
Copy link
Member

ammen99 commented Jan 29, 2024

Oh, I realized we get race conditions between the IM and the compositor. Thanks for testing this, I'll consider a bit more.

@lilydjwg
Copy link
Contributor Author

Tab should never be sent to the IM in the first place.

I realized that an alt release can also cause switcher to finish, so I'm not sure what happened. I'll do logging later.

@lilydjwg
Copy link
Contributor Author

I don't understand, but it is still the alt release event:

II 29-01-24 21:25:15.311 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2c2c940 key 56 state 1 is_im_sent false
II 29-01-24 21:25:15.312 - [build/wayfire/src/core/seat/keyboard.cpp:53] 0x557fb2c2c940 im handled above
II 29-01-24 21:25:15.312 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2ff8df0 key 56 state 1 is_im_sent true
II 29-01-24 21:25:15.313 - [build/wayfire/src/core/seat/keyboard.cpp:72] 0x557fb2ff8df0 send to surface
II 29-01-24 21:25:15.452 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2c2c940 key 15 state 1 is_im_sent false
DD 29-01-24 21:25:15.452 - [build/wayfire/src/output/output.cpp:294] output Virtual-1: activate plugin switcher
II 29-01-24 21:25:15.453 - [build/wayfire/src/core/seat/keyboard.cpp:72] 0x557fb2c2c940 send to surface
DD 29-01-24 21:25:15.453 - [types/wlr_text_input_v3.c:185] Text input commit received without focus
II 29-01-24 21:25:15.459 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2ff8df0 key 56 state 0 is_im_sent true
II 29-01-24 21:25:15.459 - [build/wayfire/src/core/seat/keyboard.cpp:72] 0x557fb2ff8df0 send to surface
II 29-01-24 21:25:15.592 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2c2c940 key 15 state 0 is_im_sent false
II 29-01-24 21:25:15.593 - [build/wayfire/src/core/seat/keyboard.cpp:53] 0x557fb2c2c940 im handled above
II 29-01-24 21:25:15.597 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2ff8df0 key 15 state 0 is_im_sent true
II 29-01-24 21:25:15.597 - [build/wayfire/src/core/seat/keyboard.cpp:72] 0x557fb2ff8df0 send to surface
DD 29-01-24 21:25:15.975 - [build/wayfire/src/output/output.cpp:311] output Virtual-1: deactivate plugin switcher
II 29-01-24 21:25:15.991 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2c2c940 key 56 state 0 is_im_sent false
II 29-01-24 21:25:15.991 - [build/wayfire/src/core/seat/keyboard.cpp:53] 0x557fb2c2c940 im handled above
II 29-01-24 21:25:15.992 - [build/wayfire/src/core/seat/keyboard.cpp:41] 0x557fb2ff8df0 key 56 state 0 is_im_sent true
II 29-01-24 21:25:15.992 - [build/wayfire/src/core/seat/keyboard.cpp:72] 0x557fb2ff8df0 send to surface

@lilydjwg lilydjwg marked this pull request as ready for review February 18, 2024 07:27
@ssfdust
Copy link
Contributor

ssfdust commented Feb 22, 2024

@ammen99 Hello, could you review this pr ? It works and solve some issues for me.

@ammen99 ammen99 added this to the 0.8.1 milestone Feb 22, 2024
@ammen99
Copy link
Member

ammen99 commented Feb 25, 2024

@lilydjwg I am looking at this again and I do not see why is_grab should be necessary. I also tried the latest fcitx5 built from git and alt-tab works correctly even without the is_grab check, i.e with Wayfire patched like the patch indicates. Note that I can reproduce the bug very easily with Wayfire-master so I think I am doing the same thing as you do to reproduce the bug. So could you please try again (or @ssfdust)? What exactly does the following patch break:

diff --git a/src/core/seat/keyboard.cpp b/src/core/seat/keyboard.cpp
index 026b4a43..e95ccc0a 100644
--- a/src/core/seat/keyboard.cpp
+++ b/src/core/seat/keyboard.cpp
@@ -65,8 +65,7 @@ void wf::keyboard_t::setup_listeners()
                 }
             }
 
-            // don't send IM sent keys to plugin grabs
-            if (seat->priv->keyboard_focus && !(seat->priv->is_grab && is_im_sent))
+            if (seat->priv->keyboard_focus)
             {
                 seat->priv->keyboard_focus->keyboard_interaction()
                     .handle_keyboard_key(wf::get_core().seat.get(), *ev);
diff --git a/src/core/seat/seat-impl.hpp b/src/core/seat/seat-impl.hpp
index f668bb31..2c00c250 100644
--- a/src/core/seat/seat-impl.hpp
+++ b/src/core/seat/seat-impl.hpp
@@ -57,7 +57,6 @@ struct seat_t::impl
 
     void set_keyboard_focus(wf::scene::node_ptr keyboard_focus);
     wf::scene::node_ptr keyboard_focus;
-    bool is_grab = false;
     // Keys sent to the current keyboard focus
     std::multiset<uint32_t> pressed_keys;
     void transfer_grab(wf::scene::node_ptr new_focus);
diff --git a/src/core/seat/seat.cpp b/src/core/seat/seat.cpp
index b47b32be..40ee3ad6 100644
--- a/src/core/seat/seat.cpp
+++ b/src/core/seat/seat.cpp
@@ -500,7 +500,6 @@ void wf::seat_t::impl::transfer_grab(wf::scene::node_ptr grab_node)
     }
 
     this->keyboard_focus = grab_node;
-    this->is_grab = true;
     grab_node->keyboard_interaction().handle_keyboard_enter(wf::get_core().seat.get());
 
     wf::keyboard_focus_changed_signal data;
@@ -522,7 +521,6 @@ void wf::seat_t::impl::set_keyboard_focus(wf::scene::node_ptr new_focus)
     }
 
     this->keyboard_focus = new_focus;
-    this->is_grab = false;
     if (new_focus)
     {
         new_focus->keyboard_interaction().handle_keyboard_enter(wf::get_core().seat.get());

Also for comparison, here is how I can reproduce the alt-tab bug on Wayfire master: https://github.com/ammen99/wayfire-tests/blob/im-tests/tests/im/fcitx5-switcher/main.py

The test runs successfully if I use this PR, but also if I apply the patch.

@lilydjwg
Copy link
Contributor Author

Can the test assert that the plugin is active before releasing the alt key?

https://github.com/ammen99/wayfire-tests/blob/81f5ff94d8cc419e94eafa5edc013087e25fe959/tests/im/fcitx5-switcher/main.py#L26

@ammen99
Copy link
Member

ammen99 commented Feb 27, 2024

Can the test assert that the plugin is active before releasing the alt key?

https://github.com/ammen99/wayfire-tests/blob/81f5ff94d8cc419e94eafa5edc013087e25fe959/tests/im/fcitx5-switcher/main.py#L26

Oh, I see, yes. Somehow this slipped my radar .. I'll think whether there is a better way to fix the problem or not. Thanks :)

Copy link
Member

@ammen99 ammen99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can merge this as it is and if I figure out a better way to fix the switcher alt-tab, I will send a follow-up PR and ping you for testing.

Thanks a lot for your work on IMs!

@ammen99 ammen99 merged commit 27ddca3 into WayfireWM:master Feb 27, 2024
4 checks passed
@lilydjwg lilydjwg deleted the im branch February 27, 2024 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants