Make PopupMenu/Panel shadows properly visible again

[YeldhamDev]

This PR does the following:

• Make PopupMenu/Panel nodes have spacing around them to show the shadows of their theming, as well as repositioning them accordingly with their shadows' size when popping up.
• Make PopupMenu/Panel nodes be transparent by default. Not only this is necessary for the shadows, but also for rounded corners. It slightly breaks compatibility, as a default value is changed. But I think it's non-intrusive enough to be acceptable.
• Fix some bugs related to content scaling, and submenu positioning.

Closes godotengine/godot-proposals#7532.
Closes godotengine/godot-proposals#8738.

Test Project: shadow-of-the-popups.zip

[Calinou]

I'd say this also closes godotengine/godot-proposals#8738 (by superseding it).

[passivestar]

Doesn't look right with the default theme on mac 🙃

[YeldhamDev]

@passivestar Could you enable display/window/per_pixel_transparency/allowed in the project settings?

[passivestar]

@passivestar Could you enable display/window/per_pixel_transparency/allowed in the project settings?

yeah that works

[YeldhamDev]

I've added a warning that appears when a PopupMenu/Panel has a theme that needs per pixel transparency but it isn't enabled.

Note that currently it always appears when running the game because the default project theme has rounded corners. If this is merged, it will need to be modified.

[bruvzg]

We probably can enable transparency for the editor by default, it may increase VRAM usage a bit, but should not have any significant performance impact on modern systems.

Also, I'll take a look if we can test if transparency is really supported and working (checks if compositor is active and surface/front buffer have correct format), and will add a method to the display server.

[YeldhamDev]

@bruvzg Nice! But I think having the warning is a good thing nonetheless.

[passivestar]

Themed the menu to take advantage of this PR, makes a huge difference imo

As I understand transparency needs to be an editor setting because we probably don't want a project setting to affect editor theming. Even better if enabled by default like @bruvzg suggested

P.S. Clicking on the popup menu shadow doesn't close the popup which is a bit annoying with a large shadow

[YeldhamDev]

P.S. Clicking on the popup menu shadow doesn't close the popup which is a bit annoying with a large shadow

Got you covered. 😉

[YeldhamDev]

Updated to use the new is_window_transparency_available() function.

[YeldhamDev]

@Geometror Huh, all backends work for me.

Anyways, CC @bruvzg, since you were the one who implemented it.

[bruvzg]

Should also work on macOS (with any renderer) and on Windows with D3D 12, but Vulkan depend on the driver and NVIDIA broke it some time ago.

[Geometror]

Alright, haven't looked at the code yet, but functionality-wise this it's almost ready - just one thing left:
This should work regardless of the backend when subwindows are embedded (plus, it should also work in the 2D editor, but I guess this comes for free when it works for embedded subwindows)

Related to this PR, but not required/just for reference:

• DisplayServer::get_singleton()->is_window_transparency_available() doesn't report per-pixel transparency for Wayland, although it is supported (in my case) as you can see in my screenshot
• Per-pixel transparency doesn't work for me on Windows 11 both with DX12 and Vulkan (single GPU system, 3060 Ti), which is kind of strange (at least when using DX12)

[YeldhamDev]

@Geometror Changes made.

[Geometror]

Works great now, nice work! Code looks fine too, with just a few consts missing :)

[YeldhamDev]

@Geometror @AThousandShips Changes made.

[molingyu]

This fix doesn't work properly when display/window/subwindows/embed_subwindows is true (not just shadows, but almost all theme settings).

It seems that for native windows, we need another set of code to handle the theme

[YeldhamDev]

@molingyu Both embedded and non-embedded works fine here. Did you enable display/window/per_pixel_transparency/allowed?

[molingyu]

@molingyu Both embedded and non-embedded works fine here. Did you enable display/window/per_pixel_transparency/allowed?嵌入式和非嵌入式在这里都工作正常。您是否启用了 display/window/per_pixel_transparency/allowed ？

No.

I used the Test Project provided by PR and the shading is still incorrect.

My environment: windows 10

[AThousandShips]

Would this be equivalent to:

Suggested change

	if (p_over.x < item_clickable_area.position.x || p_over.x >= item_clickable_area.position.x + item_clickable_area.size.width ||
	p_over.y < item_clickable_area.position.y || p_over.y >= item_clickable_area.position.y + item_clickable_area.size.height) {
	if (!item_clickable_area.has_point(p_over)) {

[molingyu]

I tried enabling display/window/per_pixel_transparency/allowed.

Shadows that extend beyond the bounds are still clipped(I used the build and test projects provided by the pr action and only changed the shadow to blue to make it easier to spot.).

By the way, my editor also became abnormally transparent.

Additional information: My graphics card is AMD Radeon(TM) Graphics

Godot v4.4.dev (dc78b4c5d) - Windows 10.0.19045 - Multi-window, 2 monitors - Vulkan (Forward+) - integrated AMD Radeon(TM) Graphics (Advanced Micro Devices, Inc.; 27.20.15026.8004) - AMD Ryzen 5 5500U with Radeon Graphics (12 threads)

I checked with other people using Windows and they have no issues.

So this may be caused by the graphics card?

At this point, pr should be fine. Could the problem come from somewhere else?

[Delsin-Yu]

cc @YeldhamDev

It looks like the issue has not been fixed in the Forward+ / Vulkan (Windows) backend when display/window/subwindows/embed_subwindows is set to false. I also get the following warning messages when launching the project under the abovementioned configuration:

W 0:00:00:0760   PopupMenu::_notification: The current theme styles PopupMenu to have shadows and/or rounded corners, but those won't display correctly if 'display/window/per_pixel_transparency/allowed' isn't enabled in the Project Settings, nor if it isn't supported.
<C++ Source>   scene\gui\popup_menu.cpp:1113 @ PopupMenu::_notification()
W 0:00:00:0762   PopupPanel::_notification: The current theme styles PopupPanel to have shadows and/or rounded corners, but those won't display correctly if 'display/window/per_pixel_transparency/allowed' isn't enabled in the Project Settings, nor if it isn't supported.
<C++ Source>   scene\gui\popup.cpp:348 @ PopupPanel::_notification()

To clarify, I do have the per_pixel_transparency enabled in the project setting, so it's somewhat not recognized by the backend.

Edit: I also ran the test on a macOS machine, and the issue in forward+ seems to be exclusive to the Windows platform.

Windows & Open GL:

Godot v4.4.dev (f7f6432) - Windows 10.0.19044 - Multi-window, 4 monitors - OpenGL 3 (Compatibility) - NVIDIA GeForce RTX 4060 Ti (NVIDIA; 32.0.15.6603) - AMD Ryzen 9 5900X 12-Core Processor (24 threads)

Embed	Not Embed

Windows & Vulkan:

Godot v4.4.dev (f7f6432) - Windows 10.0.19044 - Multi-window, 4 monitors - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4060 Ti (NVIDIA; 32.0.15.6603) - AMD Ryzen 9 5900X 12-Core Processor (24 threads)

Embed	!! Not Embed !!

macOS & Open GL:

Godot v4.4.dev (f7f6432) - macOS 15.1.1 - Multi-window, 1 monitor - OpenGL 3 (Compatibility) - Apple M4 - Apple M4 (10 threads)

Embed	Not Embed

macOS & Metal

Godot v4.4.dev (f7f6432) - macOS 15.1.1 - Multi-window, 1 monitor - Metal (Forward+) - integrated Apple M4 (Apple9) - Apple M4 (10 threads)

Embed	Not Embed

[YeldhamDev]

@Delsin-Yu The warning comes from display/window/per_pixel_transparency/allowed not being enabled by default. We should probably make so that this is always enabled for the editor itself as well in another PR.

And about it not working on some backends, it's unrelated to the PR itself, and on @bruvzg's ballpark.

[Delsin-Yu]

@YeldhamDev Just to clarify things, the warning happens exclusively on Windows & Vulkan with display/window/subwindows/embed_subwindows = false; all other working configuration sets do not have this warning.

[JekSun97]

This PR fixes my issue (#100290), I tested it in all rendering modes.
I also tested MRP.

Godot v4.4.dev unknown - Windows 10.0.19045 - Multi-window, 1 monitor - Vulkan (Forward+) - dedicated Radeon RX 560 Series (Advanced Micro Devices, Inc.; 31.0.14001.45012) - Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz (4 threads)

[bruvzg]

Just to clarify things, the warning happens exclusively on Windows & Vulkan with display/window/subwindows/embed_subwindows = false; all other working configuration sets do not have this warning.

NVIDIA broke transparency for Vulkan some time ago, I'm not sure if we can do something with it (maybe fixable by switching Vulkan to use DXGI swapchain), it's simply not supported by the current drivers. But it should work with DX12 and OpenGL.

[akien-mga]

Thanks!