Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyPressedPreferNative: More performat key input #829

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

phu54321
Copy link

@phu54321 phu54321 commented Oct 15, 2024

On windows, Gdx.input.isKeyPressed uses lwjgl's event system for fetching key inputs. This event-based system sometimes lags due to OS issues, so beatoraja had problem getting key state ASAP. When pressing multiple keys, some keys were registered later than other keys.

This commit fixes the issue by utilizing Windows's GetAsyncKeyState function for fetching current keyboard state itself.

  • Before: note that simultaneous keys are not registered simultaneously.
  • After: simultaneous keys are registered simultaneously.

Comparison before & after

Reference: previous libgdx input polling system (from LWJGL2)

Additional library:
This adds jna to the dependencies. Whether the code works on mac/linux is not yet tested. Testing is required.

https://github.com/LWJGL/lwjgl/blob/master/src/java/org/lwjgl/opengl/WindowsKeyboard.java

@phu54321 phu54321 force-pushed the feature/windows_getasynckeystate branch from e32e17a to 49b4939 Compare October 15, 2024 02:29
On windows, `Gdx.input.isKeyPressed` uses lwjgl's event system
for fetching key inputs. This event-based system sometimes
lags due to OS issues, so beatoraja had problem getting key state
ASAP. When pressing multiple keys, some keys were registered
later than other keys.

This commit fixes the issue by utilizing Windows's `GetAsyncKeyState`
function for fetching current keyboard state itself. Note that this
function will poll key state regardless of whether the window is
focused or not, so this could be documented.

Reference: previous libgdx input polling system (from LWJGL2)

https://github.com/LWJGL/lwjgl/blob/master/src/java/org/lwjgl/opengl/WindowsKeyboard.java
@phu54321 phu54321 force-pushed the feature/windows_getasynckeystate branch from 49b4939 to 8fc25ce Compare October 15, 2024 02:31
@phu54321 phu54321 force-pushed the feature/windows_getasynckeystate branch from 2481613 to 0bef0f0 Compare October 21, 2024 14:51
@phu54321
Copy link
Author

phu54321 commented Oct 21, 2024

Fix) beatoraja now only accept keys when the window is focused.

  • GetAsyncKeyState works regardless of whether the window is focused.
  • Added a check so that the key will input only when the window is focused.

…lay.isActive()

Display.isActive() requires a global lock on the display object, so it frequently stalls, resulting in less-than-optimal gaming experience.

To make things lock-free, we again use JNA to implement `Display.isActive()` ourselves.

Note that this code is coded for lwjgl2 backend the beatoraja currently uses. To maximize modularity, All input-specific code are
confined in KeyPressedPreferNative class only.
Alt+F4 should kill the application.
@Merrg1n
Copy link

Merrg1n commented Oct 26, 2024

I had noticed this issue before, which led me to write this project https://github.com/Merrg1n/beatoraja-ime-fix. After reading the source code, I found that libGDX handles keyboard input events tied to the game loop. This means the actual keyboard polling rate is bound to the game's frame rate. However, the GetAsyncKeyState API only exists on Windows, and more consideration is needed for Mac/Linux (since this project is cross-platform).

@phu54321
Copy link
Author

This means the actual keyboard polling rate is bound to the game's frame rate.


Polling happens on a separate thread, so polling rate is much faster than the display refresh rate.

Separate consideration is required for macOS/Linux. Every windows-specific aspects are enclosed on the KeyPressedPreferNative class, and support for other platforms also can be encapsulated.

@Merrg1n
Copy link

Merrg1n commented Oct 26, 2024

Polling happens on a separate thread, so polling rate is much faster than the display refresh rate.

Let me add some implementation details about LibGDX. Even though it appears that keyboard events are polled by another thread in the game, this is actually how it works.

In LibGDX 1.9.9 (the version actually used in this repository), Gdx.input.isKeyPressed() actually calls com.badlogic.gdx.backends.lwjgl.LwjglInput.isKeyPressed(). (In later versions of LibGDX, this call path has changed.)

        public boolean isKeyPressed (int key) {
		if (!Keyboard.isCreated()) return false;

		if (key == Input.Keys.ANY_KEY)
			return pressedKeys > 0;
		else
			return Keyboard.isKeyDown(getLwjglKeyCode(key));
	}

This method actually calls org.lwjgl.input.Keyboard.isKeyDown(), and then attempts to retrieve a key from the keyDownBuffer.

    public static boolean isKeyDown(int key) {
        synchronized(OpenGLPackageAccess.global_lock) {
            if (!created) {
                throw new IllegalStateException("Keyboard must be created before you can query key state");
            } else {
                return keyDownBuffer.get(key) != 0;
            }
        }
    }

So next, we should focus on where the keyDownBuffer is updated. It's actually updated in org.lwjgl.input.Keyboard.poll(), This method calls org.lwjgl.opengl.InputImplementation.pollKeyboard(), which is implemented by platform-specific classes. On Windows, it's implemented in org.lwjgl.opengl.WindowsDisplay, and ultimately, it calls GetAsyncKeyState API.

    public static void poll() {
        synchronized(OpenGLPackageAccess.global_lock) {
            if (!created) {
                throw new IllegalStateException("Keyboard must be created before you can poll the device");
            } else {
                implementation.pollKeyboard(keyDownBuffer);
                read();
            }
        }
    }

It seems like it should work correctly, but the Keyboard.poll() method is actually called within org.lwjgl.opengl.Display.pollDevices(), which is then invoked by org.lwjgl.opengl.Display.processMessages(). Finally, this brings us back to LibGDX, where it is called in com.badlogic.gdx.backends.lwjgl.LwjglApplication.mainLoop(), So, in reality, the keyboard event is polled within the main game loop.

	void mainLoop () {
                // ...
		graphics.lastTime = System.nanoTime();
		boolean wasActive = true;
		while (running) {
			Display.processMessages();
                        // ...
                }
       }

LwjglApplication.mainLoop() includes the actual rendering loop, so the polling of keyboard events is tied to the game’s rendering loop. As a result, the actual keyboard polling rate is linked to the game's frame rate.

@phu54321
Copy link
Author

Wow, awesome description :) I haven't noticed that behavior, but it seems very plausible. Probably I'll need to test it too!

My PR could also help mitigate that, as my code doesn't rely on the lwjgl's event loop system.


Minor corrections: On WindowKeyboard.poll, this code fills up the keyDownBuffer.
https://github.com/LWJGL/lwjgl/blob/2df01dd762e20ca0871edb75daf670ccacc89b60/src/java/org/lwjgl/opengl/WindowsKeyboard.java#L80C3-L80C38

key_down_buffer is filled by handleKey function, which is in turn called in response to WM_KEYDOWN and WM_KEYUP events.
https://github.com/LWJGL/lwjgl/blob/2df01dd762e20ca0871edb75daf670ccacc89b60/src/java/org/lwjgl/opengl/WindowsDisplay.java#L892

So left and right shift keys are handled with GetAsyncKeyState, but others are handled by Win32 event system. This explains why simultaneous keys were not able to be processed simultaneously.

@phu54321
Copy link
Author

phu54321 commented Oct 26, 2024

Confirmed...

Note the quantized timing graph (bottom right) when playing 180 BPM songs with 60Hz monitor vsynced game.

20241026_161914_mpc-be64_bDrk9Pclgm

Calling `Gdx.input.isKeyPressed` acquires lock of global some opengl mutex,
so it takes quite a considerable amount of time.

Filter out excessive calls with GetAsyncKeyState.
lwjgl already has implemented multiple JNIs for Windows APIs we need.
Since JNI is much faster than JNA, with a
help of various reflection help we use those
native methods instead of maintaining our own
c++/java JNI build native systems.

- Corollary: removed JNA dependencies
@phu54321
Copy link
Author

OK everything else is quite optimized, and if I replace the polling thread's Thread.sleep to Thread.yield. I can easily achieve 8k polling rate. I'm not sure whether this is the right direction.

Example implementation.

Thread polling = new Thread(() -> {
	for (;;) {
		input.poll();
		Thread.yield();
	}
});

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants