The new ClearPass leads to flickering when fps is less than 60

[folke]

Bevy version

Latest git

Operating system & version

Fedora

What you did

cargo run --example 3d_scene_pipelined

What you expected to happen

The scene should render without flickering.

What actually happened

Sometimes the scene stays blank. Other times, it flickers between the correct frame and a blank frame.

This seems to be caused by the changes in #3209

Additional information

The scene runs at 40fps on my laptop. Most time seems to be spent in light clustering:

• update_clusters: 5ms
• assign_lights_to_clusters: 9ms
• prepare_clusters: 3ms

Improving the performance of the clustering might fix this issue, but the real issue seems to be that clear happens every frame, while new frames might not be ready yet, leading to the blank frames.

If I go to a commit right before the ClearPass, then the scene renders as expected.

[mockersf]

I tried reproducing by adding thread::sleep in a user system or in system assign_lights_to_clusters, lowered FPS to 15 but no flickering issues

[folke]

The shadow pass node seems to take 4ms and runs after Clear and before Main, so maybe that's why I'm seeing flickering? And that's why it worked before?

Try adding a sleep in the shadow pass.

[folke]

The pbr_pipelined example normally runs as it should, but if I add a thread sleep of 16ms inside ShadowPassNode, then I get a black screen (in windowed mode). Whenever I move the mouse, I get flickering frames coming through.

What's weird though is that when I maximize the screen, things seem to work as expected.

Edit: still flickering on fullscreen, but much less than in windowed mode.

I have these issues on Linux both for X11 and Wayland.

[mockersf]

Can't reproduce on my mac with a sleep inside ShadowPassNode.

It may be a linux issue, or on how your card driver handles the command. Could you add the info from your gpu if someone can reproduce?

[folke]

This is on a Dell Xps 9310

AdapterInfo {
  name: "Intel(R) Xe Graphics (TGL GT2)",
  vendor: 32902,
  device: 39497,
  device_type: IntegratedGpu,
  backend: Vulkan
}

It might be a graphics card issue, but if I disable the ClearPass and do the clear as before inside MainPass3d, then it works again without flickering.

[folke]

I can reproduce the issue on my end with the following minimal example. The screen flickers very rapidly between a grey frame and a black window.

use bevy::{
    prelude::{App, Commands},
    render2::camera::PerspectiveCameraBundle,
    utils::Duration,
    PipelinedDefaultPlugins,
};

fn main() {
    App::new()
        .add_plugins(PipelinedDefaultPlugins)
        .add_startup_system(setup)
        .add_system(sleep)
        .run();
}

fn setup(mut commands: Commands) {
    commands.spawn_bundle(PerspectiveCameraBundle::default());
}

fn sleep() {
    std::thread::sleep(Duration::from_millis(20));
}

[zamazan4ik]

Do we need to test it on additional environments? I have:

• Windows 10, Nvidia videocard, proprietary driver
• Fedora 35, Tiger Lake Intel videocard
• Fedora 35, Ivy bridge Intel videocard
• Fedora 35, Nvidia videocard, proprietary driver

[folke]

Did a lot more testing and I think I'm hitting three different issues, but they give similar results:

• on Wayland, my minimal example above shows the issue. On X11 however, it works as expected. On Wayland the issue also disappears if I maximize the window. Might be fixed by an upstream wgpu PR (GLES Wayland: properly resize on configure (#2094) gfx-rs/wgpu#2284)
• on X11 I still have flickering for the 3d_scene_pipelined. That is casused by the 4ms delay between clear, shadow and mainpass. If I disable the Shadow code, the 3d scene no longer flickers. I can also trigger the issue by adding a tiny sleep inside shadow with the minimal example. This used to be fine before the ClearPass changes.
• on both Wayland and X11 in one of my projects I get a lot of flickering (which is the reason I started looking into this). But that seeems to be caused by Call to Surface::get_current_texture() seems to delay by one frame gfx-rs/wgpu#2269. I added debug prints for the duration and flickering only happens when it's over 16ms. The other two issues above are unrelated to this.

[folke]

@zamazan4ik would be great if you can confirm seeing the same behavior on your Fedora. (I'm testing on Fedora 35 with a Tiger Lake integrated GPU) Might also be good to check on Windows for sure.

[NiklasEi]

I can reproduce this on Ubuntu 18 (Intel® UHD Graphics 630). Attaching a heavily flickering run of cargo run --example 3d_scene_pipelined on latest main without any code changes.

flickering.mp4

[Weasy666]

cargo run --example 3d_scene_pipelined and the minimal example work fine on Windows 10 with a Radeon RX 570 and auto-detected vulkan backend.

[manokara]

Running 3d_scene_pipelined on a Ryzen 5 2400G APU (Vega 11) on Linux (Solus, Linux v5.14.16), using open-source amdgpu drivers.

AdapterInfo { 
    name: "AMD RADV RAVEN",
    vendor: 4098,
    device: 5597,
    device_type: IntegratedGpu,
    backend: Vulkan 
}

• Debug: No flickering, ~15 FPS (even with vsync off)
• Release: No flickering, 60 FPS (~200 with vsync off)

It could definitely be a GPU issue. From what I've heard Intel's drivers for Vulkan on Linux are not very good.

[manokara]

Ok so I just remembered I also had an Intel laptop around: An i5-3210M (HD Graphics 4000) Ivy Bridge processor running Arch Linux.

AdapterInfo {
    name: "Intel(R) HD Graphics 4000 (IVB GT2)",
    vendor: 32902,
    device: 358,
    device_type: IntegratedeGpu,
    backend: Vulkan
}

• Release: No flickering, ~24FPS

Maybe it's still an Intel issue, but only with newer iGPUs? We'll need more testing from people with Intel chips to see. This might be a stretch, but there could be other factors involved such as monitor refresh rates? Still, using the older method the flickering doesn't happen, so... stuff is happening.

[folke]

@manokara I think the main issue with ClearPass only occurs when the ShadowPass takes a noticable amount of time. This is the order of the renderpasses:

1. ClearPass: clears the entire frame
2. ShadowPass: render shadows. This pass takes 4ms for me, so the screen will be only showing shadows for 4ms
3. MainPass: here the rest of the scene is added again

Before the ClearPass PR, clearing and painting the whole scene happened at the same time.

Could you see what happens if you add a thread::sleep inside the ShadowPassNode? I don't see why this would not lead to the visible flickering. Unless I'm totally misunderstanding how all of this works, which is very likely :)

[folke]

I was able to fix the flickering for X11 by disabling vsync (either Immediate or Mailbox are fine. Fifo triggers the issue).

For Wayland, the issue still persists.

[manokara]

I assume that by "putting a sleep in ShadowPassNode" means putting it in its Node::run implementation, right? It just lowers the framerate, no flickering at all. And that wouldn't really make sense if you think about it, because the entire window goes blank.

The problem might be somewhere in the process of passing graphics data to wgpu, maybe double-buffering related. Until recently the rationale was that it's Intel related, but it works fine on my 3210M (albeit slow) as I commented before.

[cart]

This seems like a logical error somewhere below us in the stack (wgpu or a driver). The final output from our render graph is a single command queue that does clear passes first, then draws on top of them after that. Timing shouldn't come into play here because the final "command list" submitted to the gpu will be identical no matter how long each step takes.

Pulling in wgpu folks to get their take on this: @kvark and @cwfitzgerald.

[kvark]

Does it reproduce if you capture an API trace and then replay it, either locally or on a different machine?
https://github.com/gfx-rs/wgpu/wiki/Debugging-wgpu-Applications#tracing-infrastructure
Could somebody post repro instructions?

[cart]

Someone experiencing this issue can capture the trace by running the lighting example on main (where the new renderer is now the default and the 3d_scene_pipelined example has been renamed to lighting):

cargo run --example lighting --features wgpu_trace

The trace should show up in a wgpu_trace folder in the root of your project.

[folke]

@kvark I've attached two traces for the lighting example on X11 and Wayland.

Both casue the flickering (wgpu 0.11.5).

When I maximize the windows, the flickering disappears.
When I disable vsync (use Mailbox or Immediate mode), then the flickering goes away for the X11 build, but remains in the wayland build.

Thank you for looking into this!

AdapterInfo {
  name: "Intel(R) Xe Graphics (TGL GT2)",
  vendor: 32902,
  device: 39497,
  device_type: IntegratedGpu,
  backend: Vulkan
}

wayland.zip
x11.zip

[kvark]

Replaying the first capture doesn't show me any flickering. So this isn't about any kind of logical ordering issues within wgpu itself.
The only strange thing I noticed in the trace is your surface being the 2nd surface. What happened to the first? Is there any way you could test this without creating extra surfaces, just to be sure?
And again, what would be good repro steps for me?

[folke]

I don't fully understand the Bevy code, but I was able to disable the creation of the first surface, so there's only one being used. The flickering is still happening though.

@cart a surface gets created in bevy_render/lib:116 and then another one bevy_render/view/window:132.

To reproduce the issue, I clone the bevy repo and then run cargo run --example 3d_scene.

[kvark]

Running this example on Bevy revision c825fda doesn't expose any flickering to me on Linux.

AdapterInfo { name: "Intel(R) Xe Graphics (TGL GT2)", vendor: 32902, device: 39497, device_type: IntegratedGpu, backend: Vulkan }

[folke]

you have the same graphics card as I have? What distro are you running?

The results seem to be different for me whether I run the X11 build under Xorg, or under XWayland (the latter gives much more flickering). The most flickering is for Wayland builds under Wayland.

The flickering only seems to happen for apps where present_frame is not called within 16ms.

You could try the lighting example cargo run --example 3d_scene. This one is heavier and will hopefully show the flickering.

[kvark]

Yes, looks like I have the same GPU, just in a different laptop. I'm running NixOS with KDE (with X11).
VulkanInfo shows the following:

        apiVersion        = 4202678 (1.2.182)
        driverVersion     = 88088581 (0x5402005)
        vendorID          = 0x8086
        deviceID          = 0x9a49
        deviceType        = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
        deviceName        = Intel(R) Xe Graphics (TGL GT2)

Notice the driver version and see how it compares to yours.

That 3d_scene doesn't make a difference for me. Still rendering fine.

[KirmesBude]

To add on to your point of graphics card and distro: What Mesa Version are you running?

There are recent issues on the mesa repository concerning flickering on intel graphics.
https://gitlab.freedesktop.org/mesa/mesa/-/issues/5731
https://gitlab.freedesktop.org/mesa/mesa/-/issues/5744
https://gitlab.freedesktop.org/mesa/mesa/-/issues/5745

Edit: Though the flickering there seems to be different from here. So maybe unrelated.

[folke]

Nope, you're right! If I downgrade mesa-* to 21.2.3, then the flickering is gone, so definitely an issue with the new mesa drivers.

[folke]

Seems like a fix is underway https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14207

Fixes the flickering in https://gitlab.freedesktop.org/mesa/mesa/-/issues/5744

[NiklasEi]

I can reproduce that downgrading mesa to v21.2.5 fixes the issue.

Does someone have the setup to locally build and install mesa from source? It would be nice to know if https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14207 resolves our issue.

The documented build & install steps in the Readme fail for me at meson .. 😞

[folke]

@NiklasEi there's a new PR for the issue https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14237

I'll just wait it out for now.

[kvark]

I can't build this right now because it requires libdrm that is too new, but generally building Mesa was straightforward for me, so I'd like to encourage you to try building it and resolve issues on Bevy discord (can cc me if needed). It would let Intel devs know that the patch is good, or not, before merging.

[folke]

I just installed the new Fedora 35 21.3.2 build in pending/testing and everything works as expected now!

[NiklasEi]

Version 21.3.2~kisak1~b also removes all flickering for me on Ubuntu 18. The examples look so much better now 🎉