DeviceEvent should be optional, causes unneccessary CPU usage

[Ciantic]

I use winit for background application for Windows 10.

Even though not a single window is visible or activated it eats CPU if I move the mouse: Process Explorer shows 0.33% of CPU usage when moving a mouse, but when I stop the mouse it goes to 0%.

This indicates that it just keeps on listening the device events, even though I don't even need them. It's not ideal for a background application to eat CPU when mouse is moved outside the application's own window.

I don't know is background applications (that occasionally show a Window) even a target for winit, but I surely would want it to be.

Thanks.

---

There is multiple ways to fix this, easiest but not so great is to make an emit_device_events(bool) for the EventLoop builder. Another way would be to make them toggleable at runtime, so one could toggle DeviceEvents on when necessary (e.g. when the window is activated).

[alvinhochun]

Yeah, I don't quite understand why one would by default want to receive mouse movements and keyboard events when not in foreground.

Is this behaviour present on other platforms?

[chrisduerr]

There's some more discussion in #1775, but it boils down to the same issue. Specifically the mouse events are the problem, not DeviceEvents in general.

[Ciantic]

To me the biggest issue is that it winit windows also acts as keylogger, they just keeps listening keyboard events they have no business to.

[chrisduerr]

That should be irrelevant. Any application that wants to keylog on this application can, offering the ability to do so changes nothing. It's absolutely trivial to get access to. So the only thing that matters is if the application actually uses this information or not and that has nothing to do with winit providing it.

[Ciantic]

I'm pretty sure WM_KEYUP, WM_KEYDOWN, and WM_CHAR is not routed to all applications as I type right now? There is something else going on in winit than that. Each one of those messages says in Microsoft docs that:

Posted to the window with the keyboard focus

They are not globally posted by default.

Winit attaches to keyboard at lower level, and not sure why, in Windows if you use right messages you don't act as keylogger.

(To me it's odd that Windows even allows to attach to device level events without administrator powers, they just can't seem to get their act together. MacOS at least tries to handle this correctly, it should require sudo rights, and give prompts and what not.)

[garasubo]

In Windows, DeviceEvents are abstraction of Raw Input. And the document said:

By default, no application receives raw input. To receive raw input from a device, an application must register the device.

So, technically, we can make them as optional.

In X11, DevicEvents are abstraction of XI_Raw.* events. To receive them, an application need to set the mask by calling XISelectEvents. So we can make them as optional as well.

As we discuss in #1775, I agree that the functionality to disable DeviceEvents based on window status will be useful for many application, but I think providing an option to disabling DeviceEvents at all is also useful because there are applications receiving device events through WindowEvent and ignoring all DeviceEvent (e.g. Alacritty).

I can help implementing this feature for X11 once we decide what API we want to implement

[chrisduerr]

If there is a desire to disable device events completely, just having a with_device_events and set_device_events should help with both usecases. If I only want this without focus I can just call the set_device_events or whatever one wants to call it in the focus lost/gained event callbacks.

With X11's way changing this dynamically shouldn't be a problem I think, I'm not sure about changing this at runtime on Windows though.

[Ciantic]

I looked at that Raw Input WinAPI, it looks like winit registers to HID_USAGE_GENERIC_MOUSE and HID_USAGE_GENERIC_KEYBOARD.

@garasubo is there similar differentation in X11? Maybe we could have two different apis: mouse_device_events(bool) and keyboard_device_events(bool).

Btw, in Windows the winit should still listen to WM_CHAR, WM_KEYDOWN, WM_KEYUP unconditionally, they are efficient and does what they are supposed to do, they only trigger when the window is in focus etc. They need never be toggled off.

[garasubo]

is there similar differentation in X11?

Yes, we can provide API to disable mouse or keyboard independently. Currently, we set all masks to get raw events like this:

            let mask = ffi::XI_RawMotionMask
                | ffi::XI_RawButtonPressMask
                | ffi::XI_RawButtonReleaseMask
                | ffi::XI_RawKeyPressMask
                | ffi::XI_RawKeyReleaseMask;
            // The request buffer is flushed when we poll for events
            wt.xconn
                .select_xinput_events(wt.root, info.deviceid, mask)
                .queue();

So, we just need to change mask based on the application request.

[chrisduerr]

Could also just take a bitmask ourselves.

[Ciantic]

I'd like to have some discussion about this, since this is not decided yet. I think this is an issue, it depresses me slightly because it ruins the winit for me. Let me show why this is an issue:

The problem compounds really fast if you have multiple winit processes. My CPU usage goes from ~6% -> 30% just by moving mouse even though not a single winit window is visible.

I think we need someone who can decide what kind of API is wanted, the implementation would be simple if we can agree on the API.

[msiglreith]

IMO starting with an EventLoop::with_device_events(&mut self, bool) would be fine, which de/registers all raw input devices.
For game applications it's ok to receive all events and for pure UI device events are less important. If the gamepad related parts get more traction I would see that we end up with more fine-grained control in this direction (adding devices to the eventloop similar to a window).

[Ciantic]

I'm expirimenting with this kind of signature:

pub struct EventLoop<T: 'static> {
    pub(crate) event_loop: platform_impl::EventLoop<T>,
    pub(crate) device_events: bool,
    pub(crate) _marker: ::std::marker::PhantomData<*mut ()>, // Not Send nor Sync
}

impl<T> EventLoop<T> {
    /// Enable or disable device events
    pub fn with_device_events(mut self, enabled: bool) -> Self {
        self.device_events = enabled;
        self
    }
    ...
}

Since the with implies chainability, and this pattern is used e.g. in WindowBuilder. I think it's more idiomatic for winit than &mut self. I try to implement this for Windows side, maybe someone can help with the other OS's.

[rornic]

I noticed this in a project I'm working on, finding my window was was picking up keyboard events when it was in the background.

On Windows, the cause of this appears to be the RIDEV_INPUTSINK flag on this line of code. https://github.com/rust-windowing/winit/blob/v0.26.0/src/platform_impl/windows/raw_input.rs#L146.

The original PR for this code is here #482.

I'm using glutin which uses winit for windowing. I can confirm that recompiling winit with the above flag removed fixes the issue on Windows. I'm not too familiar with the Windows API though, so this change would be better made by someone who is.

[kchibisov]

The windows and x11 part of these issue were solved. Wayland doesn't have device events and macOS in a similar situation from what I know.

@madsmtm Do you know anything about device events filtering on macOS, I don't think we need it there?

[madsmtm]

On macOS, device events are only received if the application is focused, and there shouldn't be (much) additional CPU from listening to them - though for consistency, it might make sense to disable them with the filter as well.

[kchibisov]

I think this is all resolved now.