Obtaining native window handle and OS-level modal windows

[Xeverous]

I'm creating an application that (so far) is planned to be run on desktop (Linux and Windows) and the browser (via Emscripten). I need to ask the user to open a file - I'm not sure yet how such thing will be done in Emscripten build, but for Windows (or any other desktop OS) I need the native window handle.

There is SDL_Window* Magnum::Platform::Sdl2Application::window() and https://stackoverflow.com/questions/24117983/get-window-handle-of-sdl-2-application guides to this snippet:

SDL_SysWMinfo wmInfo;
SDL_VERSION(&wmInfo.version);
SDL_GetWindowWMInfo(window, &wmInfo);
HWND hwnd = wmInfo.info.win.window;

... so obtaining native window handle (at least on SDL-based applcation) is not a problem.

The questions are:

• What would be a correct place (in code) to spawn modal window? On Windows (IFileSaveDialog::Show(hwnd)), if the modal window is given the pointer to the parent window, the parent window is blocked and event handling stops there untill modal is closed because the modal temporarily takes ownership of the OS event loop - I had problems with other libraries where doing such things (spawning modal windows) inside event handling function could cause the application to stop, loop infinitely or break in some other way (any sort of unfixable event loop corruption). Usually I had to just delay spawning the modal to a different application function, outside event callbacks.
• Is there an inherent limitation that Platform::*Application can only create 1 window?
• Is there any better way to do it in Magnum? I haven't found any explicit API for OS modal windows

[mosra]

Hi!

Is there an inherent limitation that Platform::*Application can only create 1 window?

Currently, yes. There are various in-progress but stale attempts such as #168, however due to focus being elsewhere I didn't have time to make any progress there.

What people end up doing for multi-window apps instead is sidestepping the Application classes and using SDL (GLFW, ...) directly. But to be frank I don't know how modal windows are done in SDL/GLFW either, I only ever used them in Qt, and there it was as you describe -- the event loop got paused for the main window and the modal window got the input instead.

Hmm, I guess your real question is how to open a native file open/save dialog, right? That's definitely something that would be nice to have.

Getting native window handles in SDL/GLFW could be implemented in the Application classes, but I guess that alone wouldn't be as useful if you would have no working way to open dialogs from those, right?

[Xeverous]

But to be frank I don't know how modal windows are done in SDL/GLFW either, I only ever used them in Qt, and there it was as you describe -- the event loop got paused for the main window and the modal window got the input instead.

SDL does not implement modal windows at all. At most you can obtain native window handle, which is then used with system-specific APIs.

Now, about OS-specific APIs:

• Windows has actually 2 APIs of modal windows, but I get that at this point there is no incentive to support legacy dialogs - the new ones are available in Vista, 7 and later versions. New ones have pretty rich interface and are just better.
• Windows ties event loops to the window. Modal dialogs have an optional parent window parameter. If its null, the modal is a separate window (not actually modal) and both parent and "modal" independently run their event loops. If the parent window (HWND) is given, then this window is temporarily stopped from receiving events and the user can not perform actions in the parent window. In case of null HWND, the API call does not block (returns immediately).
• Windows modals require to use COM. This itself is not a big problem (MSVC does it natively and MinGW has its own "fixed/corrected/improved" headers for Windows APIs) but there are tons of small problems that need to be solved (I guess most of them are already fixed in Magnum):
  • UTF16 wchar_t or ANSII char APIs, while pretty much everyone now wants UTF-8 char or char8_t (in case of C++20). Fortunately Windows also provides built-in functions to convert unicode between UTF-8 and UTF-16.
  • There is some "registration work" required prior to performing certain API calls. The most well-known one is registering application/window classes, which I guess is done in SDL because it can create windows.
  • Some extra RAII wrappers for COM because certain code may not be exception safe so you just can't use if/goto to cleanup resources - destructors required.
  • ??? Some more "mIcRo$oFt wInDoWs" design issues like IFileSaveDialog taking FILEOPENDIALOGOPTIONS as input parameter.
• MacOS case is similar to Windows, although I'm not sure if there was a "no parent window + non-blocking" behavior support.
• There is no such thing as "GUI on GNU/Linux". Neither Linux nor GNU has uniformly accepted any UI toolkit. One needs to implement (if there is such API) modal window calls for specific toolkit, such as GTK3 and link the application to it. These are usually libraries that work on top of SDL and/or kernel-level APIs. For this case Magnum would probably need to split/derive/customize its "generic GNU/Linux/POSIX" implementation to specific toolkits.

Getting native window handles in SDL/GLFW could be implemented in the Application classes, but I guess that alone wouldn't be as useful if you would have no working way to open dialogs from those, right?

It depends, each OS/GUI-toolkit will have different requirements. IMO there should be at least 2 classes: 1 that represents OS-specific application and 1 that represents OS-specific Window. Because on many OSes certain resources are tied to each window and other certain ones to the application as a process. Modal functionality would be in window class. Splitting resource lifetime per application and window class is the cleanest solution I can think of.

In the case of modals - I have used tinyfiledialogs project which "magically" has no dependencies and implements typical blocking modal on each platform, without even requiring to link anything (except on Windows). I guess it calls shell utilities or does dlopen sort of stuff.

---

Last things: to help you I can:

• Share a spreadsheet that compares modal window APIs on different platform. Basically tabelarized notes from recent time when I was contributing to elements library.
• Give a commit link how I have integrated tinyfiledialogs into my project (which has a Magnum/ImGui executable).

[mosra]

at this point there is no incentive to support legacy dialogs - the new ones are available in Vista

Yeah I don't intentionally support XP either anyway, so this should be fine.

wchar

Yep, there's Utility::Unicode::widen(), which wraps the Windows APIs (and thus is Windows-specific because no other platform really needs this anyway) and which is used by all Magnum APIs that deal with Unicode on Windows.

tinyfiledialogs

That's exactly what I wanted to base the implementation on / steal from -- doing some dlopen() in case the API isn't implicitly linked from elsewhere. Actually, it was this instead: https://github.com/mlabbe/nativefiledialog

UI on Linux

I'm aware :) I'd go with KDE and GTK support. With KDE it'd be I assme rather easy, unfortunately as far as I know with GTK there's no way to unload the library so once it gets loaded it'll be in memory forever. I think because of this SDL had to use some sort of IPC to avoid loading GTK in SDL_ShowMessageBox (can't find the source tho).

Give a commit link how I have integrated tinyfiledialogs into my project

Yeah that would be great. But please note that I'm still not able to invest any time into this myself in the near future.

[Xeverous]

That's exactly what I wanted to base the implementation on / steal from -- doing some dlopen() in case the API isn't implicitly linked from elsewhere.

Then tinyfiledialogs should be the prefferred choice. I have searched for many similar solutions (found also the one you linked) and this one has least dependencies, is completely troubleless, no linking required (except on Windows to some system DLLs), no global state init required, fully C and C++ compliant, single file implementation ... basically ideal.

with GTK there's no way to unload the library so once it gets loaded it'll be in memory forever

Ahh, I remember similar nightmare trying to RAII-ize fontconfig library. It has one of the worst C library designs I have seen when it comes to initialization:

• There are 3 different initialization ways; one depends on reading external XML file.
• There is no way to check if library has been initialized.
• Any extra initialization attempt is a no-op.
• Any extra cleanup attemt is undefined behavior.

A single project can deal with it. But when your project has 2 dependencies and both of them try to correctly use fontconfig, it is a distaster. The only way to "fix" it is to force a leak and make sure fontconfig is never attempted to be cleaned up twice. I just can't get why it is so complex, why there is no init check and why extra init/cleanup usage is inconsistent. Many libraries have just an integer reference counter or simply create 2 independent instances.

Give a commit link how I have integrated tinyfiledialogs into my project

Yeah that would be great. But please note that I'm still not able to invest any time into this myself in the near future.

Fine, I already see you must be busy by looking at the reponse times on this issue. I'm currently more involved in using ImGui than Magnum.

Xeverous/filter_spirit@3e06e81

This commit is actually bigger than I thought (it also adds stuff to my project that immediately uses it) so I think a more comfortable way would be to state that everything is in this path: https://github.com/Xeverous/filter_spirit/tree/3e06e81a40c9655fddbb8d1a9cefbd6fb81ed5e4/external/tiny_file_dialogs - the whole thing is just a single CMake file + subproject link (link is not clickable because tinyfiledialogs project is not located on github).