Backends: Vulkan: Make descriptor pool optional

[zeux]

Instead of specifying DescriptorPool when initializing Vulkan backend, it is now possible to instead specify the descriptor pool size. An internal descriptor pool will be created and destroyed alongside other objects.

The size needs to accommodate the textures that will be added, thus it will be application dependent.

This makes integration into Vulkan renderers simpler. Not all renderers have an available descriptor pool with _FREE_DESCRIPTOR_SET bit that is not reset every frame; right now in many cases integration will require creating a separate pool and destroying it alongside imgui backend. With this change, the process is simplified by specifying the maximum number of descriptors.

See also: #4867

[ocornut]

Thanks Arseny. This coincide with work I've been doing which will require me to settle on a few related things.
May I stray on a few tangents to get your advice here?

TL;DR I am working on a small revamping of IO to allow imgui to create and update textures (for dynamic font atlas support etc.). As a result the backend will need (occasionally, and generally temporarily) to have multiple textures allocated, e.g. when needing to grow a texture we keep the previous one (for previous in-flight renders + for what has been submitted so far in the frame) and we create a second texture.

I've implemented this new API in the DX12 backend on my WIP branch. I came up with this earlier change in public backend API: 40b2286 (+ minor tweaks 08400f5, 142827f). Basically the DX12 backend user can pass alloc/free callbacks for when the backend needs a SRV descriptor which is currently our way to describe an image when e.g. using ImGui::Image(). Following this change, in 40b2286 the examples's main.cpp provide a simple allocator (basically a free list), which is supposed to mimic what a real DX12 application would use.

A. If you are familiar with DX12, do you think my change above is a viable design?
B. I will need to implement the texture stuff for Vulkan, but if I understand things correctly (and pardon me but I'm not a real user of neither DX12 neither Vulkan), the ExampleDescriptorHeapAllocator in DX12 example has no equivalent as Vulkan land as the VkDescriptorPool serves the equivalent purpose? And therefore there will be no need to add alloc/free callback, just the existing descriptor pool (+ your patch to make it optional).

Thanks!

[zeux]

Re: B: Yeah this is largely correct. There is a way to allocate descriptors from a heap manually in Vulkan when you use EXT_descriptor_buffer extension, but this extension is very optional (not implemented/supported on some hardware and drivers), and requires different setup in some places. In general in Vulkan the descriptor story is complicated as there's a few different options to manage them (e.g. descriptors don't necessarily need to be allocated at all when using KHR_push_descriptors), the only actual constant right now is descriptor pools (from Vulkan 1.0) and the newer mechanisms are in flux. So I would recommend to stay with pool allocation for now.

Re: A: Just so that I understand better, before DX12 backend only ever needed 1 SRV, but you want to allow using more than 1? How high do you want that number to be? The alloc/free callbacks will work. They also might be excessive: I think an SRV descriptor is something like 32 bytes of space in the resource descriptor heap. If you can eventually need thousands, or if the number is not bounded a-priori (for example, the UI allows the app to insert new generated textures into a list), then you want something very dynamic like the scheme you've added. In this case Vulkan scheme might need to also be reworked eventually, because you'd need an ability to allocate new pools as each pool can only allocate a finite number of descriptors. If you only need a few, however, then the alloc/free callbacks will work but it feels like you could also change the interface in a simpler, more compatible, way by asking the user to provide N descriptors (and the number N which needs to be within some bounds) instead of alloc/free callbacks. But your scheme will work as well of course, so that's up to you.

[ocornut]

A: Just so that I understand better, before DX12 backend only ever needed 1 SRV, but you want to allow using more than 1?

Yes.

How high do you want that number to be?

In my current work branch it can never technically be more than (1+in_flight_frames) simultaneous textures in the worst case scenario, so a very small number. But that's without using user supplied images (currently requiring ImGui_ImplVulkan_AddTexture() before any ImGui::Image() call, which is possibly awkward).

I have squashed and merged your PR (with minor amends) as 61ab94d.
I added extra comments + changelogs + made the Init function assert that either input are set but not both.

Thanks for your feedback, this is helpful. My recent journey into DX12 stuff made me better understand a few more things about the Vulkan backend.

[ocornut]

While this is fresh for you, do you have an opinion on this (old) suggestion to:

• change ImTextureID to pass an vkImageView (instead of a descriptor)
• let the backend completely handle and manage its own descriptor pool dynamically?

It has been implemented in this commit: (it's old so probably conflicts)
8478cc6

Currently we bundle more information into ImTextureID (effectively, our descriptor set currently bundles Sampler + ImageView + ImageLayout).

Another approach would be to bundle less, make ImTextureID a lower-level type e.g. vkImageView, make the backend dynamically manages descriptor sets, and facilitate the use of ImDrawList::AddCallback() to change some of those callbacks (similar to https://github.com/ocornut/imgui/wiki/Image-Loading-and-Displaying-Examples#modifying-render-state). So eg. a same vkImageView may be rendered with different samplers.

We haven't nailed please-everyone solution for Vulkan at the moment (partly due to lack of trying, much confusion on my end, and a lesser need as many Vulkan users end up wrapping and using a custom renderer anyhow) but I'd eventually like to.

[zeux]

I don't have a super solid opinion on this as I don't fully understand the use case wrt callbacks here, but in general I would note that in Vulkan, it's difficult to compose render state as the resource binding is tied to the pipeline state and it's hard to decouple.

So:

• If the backend wants to just internally manage descriptor pools, this certainly seems fine; in fact, that's basically the only way to improve backwards compat through changes like the next one as a user-specified descriptor may not have the relevant resource slots!
• Using separate images & samplers is also completely fine, and if there is a desire to render the same image with different filtering settings, that's definitely how it should be done;
• ... but I guess I don't know if there is a strong reason to change the current approach, as I don't know a lot about ImGui feature set...
• ... and I would generally expect that this change should be done if the core of ImGui wants some feature the Vulkan backend doesn't provide, as opposed to allowing some functionality to be pluggable by the user, as it seems like the margin where a custom renderer is too much work but the built-in backend is not enough is rather slim.