iOS touch events scale issue

[hateom]

Hi,

I just tried to use sokol in iOS (iPad) and I am noticing that touch events (ImGui::GetMousePos()) returns values scaled by 0.5f. As if the DPI scaling is not properly calculated between the screen size and the input position. When I scale the width/height passed to the simgui_frame_desc_t then it's correct, but obviously it renders just the quarter of the screen.

What am I missing?

Also, there's a crash when I try to change the orientation of the screen, but this seems to be only a problem in the simulator, not on the real device.

Tom

[floooh]

First: Are you using the GL or Metal backend? The GL backend on iOS may suffer from problems because GL has been deprecated, so this probably won't see much fixes in the future. If it's the Metal backend, it might be a bug though.

Second, are you properly communicating the dpi scale from sokol_app.h to sokol_imgui.h (via simgui_frame_desc_t.dpi_scale = sapp_dpi_scale();?

(e.g. see the entry for 18-Jan-2022 in the changelog: https://github.com/floooh/sokol/blob/master/CHANGELOG.md)

This part of the code (reporting the dpi scale each frame instead of just at startup also had changed quite recently, but this was mainly for macOS to deal with multi-monitor scenarios, might be that the iOS sokol_app.h backend doesn't handle this correctly for some reason).

Third: what combination of iOS version and Xcode version are you using?

I'll try to look at the touch position problem in the evening on my iPad Mini4.

As for the crash on rotation, I recently fixed a similar problem (for Metal), see this ticket:

#645

Are you on sokol_app.h version which contains this fix?

Cheers!

[hateom]

Thanks for the quick reply!
Using Metal backend (pretty much the same code works well on macOS).

Here's how I set the dpi for ImGui:

        simgui_frame_desc_t imgui_frame_desc{};
        imgui_frame_desc.width = sapp_width();
        imgui_frame_desc.height = sapp_height();
        imgui_frame_desc.delta_time = sapp_frame_duration();
        imgui_frame_desc.dpi_scale = sapp_dpi_scale();
        simgui_new_frame(imgui_frame_desc);

I'm at latest Xcode (13.3.1) and latest iOS (15.4.1).

Regarding crash - I have updated the repo and the issue is indeed gone, thanks!

[floooh]

Oki doki, then I shall try to look into the touch pos issue in the evening :)

[floooh]

Ah damn, I had already forgotten the sort of trouble I had with on-device development :D

• my old Mac (which has USB-A connectors) doesn't support the latest Xcode anymore, which is needed to build apps for the iOS version I have on my iPad
• but I can't connect the iPad to my new Mac because I don't have a USB-A to USB-C adapter
• my old development team id and code signing certificate has expired because I changed companies, not a big deal I just need to update my account settings

I'll see if I can reproduce the problem on the simulator, and I'll try to get a adapter tomorrow.

Btw, Dear ImGui has some known problems with touch input: some widget types (for instance the menu) requires hover-information in the frame before a "click" happens. A touch interface doesn't provide such hover information so some of the UI elements are broken.

[floooh]

Just for me understanding the problem correctly: pressing a button or similar in the Dear ImGui UI works, right? Because that's what I see on the simulator. Some operations like dragging windows doesn't work great, but the touch coordinates that ImGui uses for its own "hit tests" seem to be alright.

However, now that I read your bug description carefully again, I noticed that you mention ImGui::GetMousePos(). That this function reports the "wrong" coordinates is actually correct, because the way sokol_imgui.h works is that Dear ImGui is blissfully unaware of any low- vs high-dpi differences.

E.g. look here, the position that's reported to Dear ImGui is the sokol_app.h mouse- or touch-position divided by the dpi_scale value (see the call to _simgui_add_mouse_pos_event()):

sokol/util/sokol_imgui.h

Lines 2197 to 2200 in 9a6237f

	case SAPP_EVENTTYPE_MOUSE_DOWN:
	_simgui_add_mouse_button_event(io, (int)ev->mouse_button, true);
	_simgui_add_mouse_pos_event(io, ev->mouse_x / dpi_scale, ev->mouse_y / dpi_scale);
	_simgui_update_modifiers(io, ev->modifiers);

(the same is true for any touch-input positions).

...so if you get the mouse/touch position via ImGui::GetMousePosition() you actually need to multiply it with sapp_dpi_scale() to convert back to the sokol-app mouse/touch position, that's intended.

A better alternative to ImGui::GetMousePosition() would be reading the mouse- or touch-positions directly from the sokol-app input events, you just need to be aware that sokol-app doesn't do any sort of input merging, e.g. touch input and mouse input is strictly separated.

Please let me know if this cleans up the confusion, or if there's still an actual bug on a real device (as I said I can't test on a real device until tomorrow or so).

Cheers!

[hateom]

Ohhhh, you are absolutely right.
I was confused by the values I was getting from GetMousePos() and the fact that dragging the windows was a bit weird - the window jumps when I try to reposition in.
But you are right - clicking buttons seems to be OK - so it's my bad.

Thanks for the explanation!

While we are at it - in Xcode I am getting this message in the debugger window at startup:

2022-04-29 22:22:38.655688+0200 app[76485:13854814] [UIFocus] Failed to update focus with context <UIFocusUpdateContext: 0x6000016a0320: previouslyFocusedItem=(null), nextFocusedItem=(null), focusHeading=None>. No additional info available.

and the app crashes when I close the simulator in _sapp_ios_run at UIApplicationMain - SIGTERM. Is it something that I should be worried about?

[floooh]

Yes I'm seeing those too, I'm not too concerned. The debug console warnings seem to change with each new macOS/iOS or SDK version, some of those are deep in the callstack of OS functions, so it's sometimes hard to make sense of them or figure out if they are caused by the application code or something else inside the OS.

The crash seems to be because the simulator doesn't seem to properly stop applications before it shuts down.

PS: closing the ticket now :)