Improve macOS mouse latency and stale positioning

[randrew]

Prior to this patch, _sapp_macos_update_mouse() used [window mouseLocationOutsideOfEventStream] to get the mouse position. mouseLocationOutsideOfEventStream seems to often lag behind the position of the mouse in the application's windows as reported by incoming mouse NSEvents.

Because the old code before this commit only called _sapp_macos_update_mouse() right before _sapp_frame(), the individual mouseMoved:, mouseDragged:, etc. methods on the _sapp_macos_view class, which are the only entry points for calling _sapp_macos_mouse_event() to send the mouse data to the user application, would be sending the mouse position data that was returned by _sapp_macos_update_mouse(), not by the event in the incoming NSEvent.

Let's review what this looks like:

• _sapp_macos_update_mouse() is called right before rendering a frame. [window mouseLocationOutsideOfEventStream] is called within, and its return value is used to update the mouse position in the global _state. This data is not dispatched to the user application here. Additionally, mouseLocationOutsideOfEventStream may have returned a laggy mouse position.

• _sapp_frame() is called and a frame is rendered.

• Time passes.

• NSEvents arrive on the mouseMoved:, mouseDown:, etc. methods. The mouse positions within the events are ignored, and instead, the position stored in the global _state from the previous call to _sapp_macos_update_mouse() is sent to the user application.

• Time passes.

• _sapp_macos_update_mouse() is called, …

This would result in the mouse input being delayed by one or more frames (at 60hz) on top of any additional latency added by vsync timing, video buffering, and display lag.

Another problem: after the user brings their mouse to a stop, the mouse position will often be incorrect — the newest mouse position has yet to be processed by the application. You can reproduce by moving the mouse quickly in a test application that draws a trail of squares along the mouse positions. When moving the mouse quickly and then coming to a stop, the trail of squares will stop short of the mouse cursor. Even if the mouse is left unmoving for seconds or minutes. Then the next time the user moves the mouse by even 1 pixel, a square will finally be drawn underneath the mouse position. (Except it will actually be the previous, old position.)

---

This commit makes several changes to input event handling in macOS:

_sapp_macos_update_mouse() has been changed to take an NSEvent* argument, instead of no arguments. And instead of using [window mouseLocationOutsideOfEventStream] to get the position of the mouse, it uses event.locationInWindow. This seems to sometimes eliminate a frame of mouse latency.

_sapp_macos_update_mouse() is no longer called immediately before rendering a frame. Instead, it's called at the beginning of each of the mouseMoved:, mouseDragged:,mouseDown:, etc. methods with its associated NSEvent*. Immediately after, _sapp_macos_mouse_event() is called. This eliminates a full frame of mouse latency. This also fixes situations where the mouse cursor's resting apparent position would lag behind its resting true position after the user stops moving the mouse.

_sapp_macos_poll_input_events() has been added, which calls [NSApp nextEventMatchingMask: ...] to manually read and dispatch input events from the event queue. _sapp_macos_poll_input_events() is called immediately before _sapp_macos_frame(). This will sometimes catch and dispatch input events that occurred shortly before the frame is to be rendered, but hadn't yet been dispatched by other mechanisms. This occasionally eliminates a frame of latency, and improves the worst-case bounds. This seems to happen to 5% or fewer mouse events.

[NSEvent setMouseCoalescingEnabled:NO], if it even still does anything, is now called once at during application startup instead of toggled off and on along with mouse lock.

---

The improvements are noticeable in both Metal and OpenGL.

This significantly improves #341 and #415 for Mac.

[randrew]

Disclaimer: I have only tested this on 10.12. But I would be surprised if it doesn't work in all later versions.

[floooh]

Looks like a good fix, but I first need to do some git archeology to find out why mouseLocationOutsideOfEventStream is used in the first place instead of the more straightforward mouse movement events, and I need to do some testing. So it'll take a few days.

I'll start looking into this now though.

[floooh]

Ok, merged! Just wanted to mention that this was a really nice PR, very non-intrusive, nicely written and explained. Thanks :)

[floooh]

Hmm, I'm still seeing infrequent odd behaviour when pressing the window close button. Sometimes the close button remains pressed, rendering stops, and the application doesn't close.

I'll remove the event polling for a little while and see if that makes any difference. Just FIY.

(PS: I'm seeing this odd behaviour both with GL and MTKView - there's a small chance that it's caused by the macOS Beta though).