Add new "input override" system for TAS input and Android touch controls #9624

JosJuice · 2021-04-05T08:13:33Z

Dolphin currently has two parts of the codebase that directly map UI elements to emulated controls: TAS input in DolphinQt, and the touch overlay on Android. The two accomplish this using entirely different methods, both of which have their own drawbacks. TAS input manually pokes around inside Wii Remote input reports instead of letting this be handled by the normal input report creation code, and does so in a way that isn't always reliable. The Android touch overlay requires a fixed mapping to be present in GCPadNew.ini and WiimoteNew.ini, meaning that these two files can't be used for storing the user's actual controller mappings, which has led to the creation of a second controller mapping system implemented in Java which lacks many of the features present on PC.

This pull request tries to improve the situation by replacing both of these two systems with a new "input override" framework. The basis of it is that each EmulatedController lets you set a callback function which can inspect and replace the input coming from InputCommon. TAS input and Android both register such callback functions.

Getting rid of the Android touch overlay's dependency on ButtonManager like this PR does is a prerequisite for getting rid of the controller indirection on Android. I'm aiming to work on fully getting rid of the controller indirection around July if this gets merged.

One important note: This pull request breaks a certain hack that some Android users have been using in which they intentionally desync the controller used by the emulation core and the controller used by the touch overlay by first configuring one Wii Remote extension in a game INI and then configuring a different Wii Remote extension in the touch overlay settings. Normally this would result in the touch overlay not working, but what these Android users then do is to edit the fixed mappings in WiimoteNew.ini (typically by using a profile instead of by editing the file directly) to let them play games that use a Wii Remote or Wii Remote + Nunchuk using an on-screen Classic Controller, which has more buttons. (This is especially popular for Skyward Sword, because of its complex motion controls.) With this PR, the contents of WiimoteNew.ini no longer affect the touch controller, so this "trick" no longer works. Since editing the fixed mappings in WiimoteNew.ini isn't something you're intended to do, and since this PR is necessary for future improvements of the controller code on Android, I am not going to try to support this hack. The proper way to go would be to add a way for users to add custom touch overlay buttons with custom mappings, but that would have to be a future project.

@jordan-woyak I would appreciate if you could review this. Not sure if this approach is the best one possible, but I would really like to be able to get rid of the controller indirection on Android.

Source/Core/Core/HW/GCPadEmu.h

Rumi-Larry · 2021-04-25T20:02:29Z

Why the terms "Main Stick" and "C-stick" not translatable?

JosJuice · 2021-04-25T20:04:41Z

The translatable terms are "Control Stick" and "C Stick". See PR #3033.

Rumi-Larry · 2021-04-25T20:40:09Z

The translatable terms are "Control Stick" and "C Stick". See PR #3033.

Interesting, thank you!

Source/Core/Core/HW/GCPadEmu.cpp

Source/Core/Core/HW/WiimoteEmu/Extension/Nunchuk.cpp

Source/Core/InputCommon/ControllerEmu/ControlGroup/ControlGroup.h

Source/Core/InputCommon/ControllerEmu/ControlGroup/Cursor.cpp

AdmiralCurtiss · 2022-04-10T01:26:36Z

The rest of the non-Android code looks fine from what I can tell, though I'm not super familiar with the input code.

JosJuice · 2022-04-20T21:34:50Z

Fixes https://bugs.dolphin-emu.org/issues/12894.

AdmiralCurtiss · 2022-04-20T23:50:52Z

Can we get a review from @jordan-woyak on this? I'd like to see this merged so we can inch closer to fixing that ridiculous Android input system.

I suspect this can also be leveraged to improve (non-GC) controller behavior on netplay...

saminamo

Looks good to me

jordan-woyak

Code looks good. Untested, but LGTM.

Like 3bc4968 but for Wii Remote extensions. I'm doing this to ensure that TAS input values will still roundtrip after the next commit.

This lets the TAS input code use a higher-level interface for overriding inputs instead of having to fiddle with raw bits. WiiTASInputWindow in particular was messy with how much controller code it had to re-implement.

This is the first step of getting rid of the controller indirection on Android. (Needing a way for touch controls to provide input to the emulator core is the reason why the controller indirection exists to begin with as far as I understand it.)

JMC47 · 2022-10-03T20:25:33Z

whelp, I see approval, I see green button.

Pokechu22 · 2022-10-04T04:33:07Z

Source/Core/DolphinQt/TAS/TASInputWindow.cpp

@@ -151,7 +216,6 @@ QSpinBox* TASInputWindow::CreateSliderValuePair(QBoxLayout* layout, int default_
  slider->setRange(0, max);
  slider->setValue(default_);
  slider->setFocusPolicy(Qt::ClickFocus);
-  slider->setInvertedAppearance(invert);


Why was this (and the invert parameter in general) removed? The IR widget used them. (However, it seems like things still behave correctly - the sliders for the IR widget still match the position and aren't reversed, at least on my machine, so I'm not sure what it does; the docs for setInvertedAppearance don't really clarify either.)

Previously, the IR sliders were more or less directly mapped to the positions of the sensor bar lights from the perspective of the IR camera. When you move a Wii Remote right, the lights appear to move to the left from the perspective of the camera – so TAS input was designed so that when you move the slider right, we invert the input so that the light points actually move to the left. With the new input override API, the sliders are mapped to the Point action, and WiimoteEmu takes care of translating from "location you're pointing at" to "light locations".

Pokechu22 · 2022-10-04T04:33:52Z

Source/Core/DolphinQt/TAS/IRWidget.h

 // Should be part of class but fails to compile on mac os
+static const u16 ir_min_x = 0;
+static const u16 ir_min_y = 0;
 static const u16 ir_max_x = 1023;
 static const u16 ir_max_y = 767;


Note that these can be in the class (at least if they're changed to constexpr). This is a change I make in the balance board PR (#8268).

see dolphin-emu#9624 for details on the rewrite. We now basically get per-button control for all supported input methods for free. Fixes for documentation and python stubs pending. Breaking changes!

JosJuice force-pushed the input-override branch 3 times, most recently from 2563bc3 to 8fc82f0 Compare April 5, 2021 11:31

JosJuice mentioned this pull request Apr 22, 2021

Dolphin iOS #9662

Closed

jordan-woyak self-requested a review April 25, 2021 17:27

Rumi-Larry reviewed Apr 25, 2021

View reviewed changes

Source/Core/Core/HW/GCPadEmu.h Outdated Show resolved Hide resolved

JosJuice force-pushed the input-override branch 2 times, most recently from a593ecc to 5c4cb92 Compare April 25, 2021 19:19

JosJuice force-pushed the input-override branch 2 times, most recently from 6794140 to 09720bf Compare April 27, 2021 22:02

JosJuice force-pushed the input-override branch from 09720bf to 3e0af3d Compare May 14, 2021 10:41

JosJuice force-pushed the input-override branch from 3e0af3d to 2c4de0b Compare June 30, 2021 07:10

JosJuice force-pushed the input-override branch 4 times, most recently from f28c8bc to 60fc3a7 Compare August 14, 2021 13:08

JosJuice mentioned this pull request Sep 19, 2021

DolphinQt: Reduce latency of TAS input's controller input passthrough #10113

Merged

JosJuice force-pushed the input-override branch from 60fc3a7 to 093e7ab Compare September 20, 2021 14:00

JosJuice force-pushed the input-override branch from 093e7ab to 5cef0c2 Compare December 12, 2021 15:53

JosJuice force-pushed the input-override branch 2 times, most recently from 53c19ea to 1198d4e Compare December 24, 2021 19:59

JosJuice force-pushed the input-override branch from 1198d4e to 7baa265 Compare January 1, 2022 18:53

JosJuice force-pushed the input-override branch from 7baa265 to 6418f36 Compare January 29, 2022 11:59

JosJuice force-pushed the input-override branch from 6418f36 to 7c562c3 Compare February 5, 2022 14:20

JosJuice force-pushed the input-override branch from 7c562c3 to 5018ab7 Compare March 15, 2022 16:45

AdmiralCurtiss reviewed Apr 10, 2022

View reviewed changes

JosJuice force-pushed the input-override branch from 5018ab7 to 245aa50 Compare April 10, 2022 08:14

JosJuice force-pushed the input-override branch from 245aa50 to ad1dbdb Compare April 17, 2022 12:42

JosJuice force-pushed the input-override branch from ad1dbdb to 82503c3 Compare May 8, 2022 09:59

JosJuice force-pushed the input-override branch from 82503c3 to 2457f8b Compare June 12, 2022 10:55

JosJuice force-pushed the input-override branch from 2457f8b to cfd106d Compare June 28, 2022 22:25

JosJuice force-pushed the input-override branch from cfd106d to 82e749d Compare July 20, 2022 14:47

saminamo approved these changes Jul 23, 2022

View reviewed changes

JosJuice force-pushed the input-override branch from 82e749d to 5ccd50b Compare August 4, 2022 07:03

JosJuice force-pushed the input-override branch 2 times, most recently from 76f62be to 437917b Compare September 23, 2022 15:40

JosJuice force-pushed the input-override branch 2 times, most recently from 2b9ce8d to 662ed7e Compare September 29, 2022 16:41

jordan-woyak approved these changes Oct 3, 2022

View reviewed changes

JosJuice added 4 commits October 3, 2022 22:00

ControllerEmu: Add new "input override" system

cb16d20

ControllerEmu: Round input floats instead of casting, part 2

8fd2525

Like 3bc4968 but for Wii Remote extensions. I'm doing this to ensure that TAS input values will still roundtrip after the next commit.

DolphinQt: Use input override system for TAS input windows

b296248

This lets the TAS input code use a higher-level interface for overriding inputs instead of having to fiddle with raw bits. WiiTASInputWindow in particular was messy with how much controller code it had to re-implement.

JosJuice force-pushed the input-override branch from 662ed7e to 51ee05c Compare October 3, 2022 20:04

JMC47 merged commit 052c739 into dolphin-emu:master Oct 3, 2022

JosJuice deleted the input-override branch October 3, 2022 20:27

Pokechu22 reviewed Oct 4, 2022

View reviewed changes

Pokechu22 mentioned this pull request Oct 4, 2022

[WIP] Balance Board emulation #8268

Draft

JosJuice mentioned this pull request Dec 27, 2022

Android input overhaul #11385

Merged

10 tasks

Medard22 mentioned this pull request Jul 25, 2023

Integrate various small PRs - mostly unrelated to Android Medard22/Dolphin-MMJR2-VBI#55

Merged

Felk mentioned this pull request Sep 24, 2023

TAS input window breaks when running scripts and vice versa Felk/dolphin#44

Open

Medard22 mentioned this pull request Dec 14, 2024

Core/Movie: Refactor to class, move to System. Medard22/Dolphin-MMJR2-VBI#830

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new "input override" system for TAS input and Android touch controls #9624

Add new "input override" system for TAS input and Android touch controls #9624

JosJuice commented Apr 5, 2021 •

edited

Loading

Rumi-Larry commented Apr 25, 2021

JosJuice commented Apr 25, 2021

Rumi-Larry commented Apr 25, 2021 •

edited

Loading

AdmiralCurtiss commented Apr 10, 2022

JosJuice commented Apr 20, 2022

AdmiralCurtiss commented Apr 20, 2022

saminamo left a comment

jordan-woyak left a comment

JMC47 commented Oct 3, 2022

Pokechu22 Oct 4, 2022

JosJuice Oct 4, 2022 •

edited

Loading

Pokechu22 Oct 4, 2022

Add new "input override" system for TAS input and Android touch controls #9624

Add new "input override" system for TAS input and Android touch controls #9624

Conversation

JosJuice commented Apr 5, 2021 • edited Loading

Rumi-Larry commented Apr 25, 2021

JosJuice commented Apr 25, 2021

Rumi-Larry commented Apr 25, 2021 • edited Loading

AdmiralCurtiss commented Apr 10, 2022

JosJuice commented Apr 20, 2022

AdmiralCurtiss commented Apr 20, 2022

saminamo left a comment

Choose a reason for hiding this comment

jordan-woyak left a comment

Choose a reason for hiding this comment

JMC47 commented Oct 3, 2022

Pokechu22 Oct 4, 2022

Choose a reason for hiding this comment

JosJuice Oct 4, 2022 • edited Loading

Choose a reason for hiding this comment

Pokechu22 Oct 4, 2022

Choose a reason for hiding this comment

JosJuice commented Apr 5, 2021 •

edited

Loading

Rumi-Larry commented Apr 25, 2021 •

edited

Loading

JosJuice Oct 4, 2022 •

edited

Loading