Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Surface::get_current_texture does not block when the window is fully invisible (macOS) #4779

Open
ComfyFluffy opened this issue Nov 26, 2023 · 7 comments

Comments

@ComfyFluffy
Copy link
Contributor

ComfyFluffy commented Nov 26, 2023

Description
I am currently following a WGPU Tutorial and have implemented Hello Triangle with vertex buffers.
image

When I click the green button to expand the window into fullscreen, the process will suddenly take up to 22GB of memory:
image

After a few seconds, the memory usage goes down:
image

image

Repro steps
I have uploaded the code onto GitHub.

git clone https://github.com/ComfyFluffy/wgpu-tutorial.git
cd wgpu-tutorial
RUST_LOG=info MTL_HUD_ENABLED=1 cargo run

Then expand the app to fullscreen.

If expanding to fullscreen does not cause memory peak, switching to other desktop then switch back to the app can also cause it.

Resizing on windowed mode does not trigger the behavior.

Resizing on Windows 11 does not trigger the behavior.

Expected vs observed behavior

Expected: The memory usage should remain largely the same during resizing.
Observed: The memory usage goes up to 22 GB.

Extra materials
Info log logged by env_logger. Memory peak happened during resizing.

[2023-11-26T06:43:52Z INFO  wgpu_core::instance] Adapter Metal AdapterInfo { name: "Apple M1 Pro", vendor: 0, device: 0, device_type: IntegratedGpu, driver: "", driver_info: "", backend: Metal }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: Fifo, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:52Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:53Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:53Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:53Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1256, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:56Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1256, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 5120, height: 2880, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:56Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 5120, height: 2880, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }

Platform
Macbook Pro 14 (2021), M1 Pro with macOS Sonoma, 16 GB Memory
wgpu: 0.18.0

@ComfyFluffy
Copy link
Contributor Author

ComfyFluffy commented Nov 26, 2023

Setting to fullscreen with window.set_fullscreen(Some(winit::window::Fullscreen::Borderless(None))) (winit window) on macOS will also cause the issue, but not on Windows 11.

The issue does not present when using Vulkan with MoltenVK.

@cwfitzgerald
Copy link
Member

Try again now that #4781 has landed

@cwfitzgerald
Copy link
Member

This doesn't reproduce for me basically identical setup on trunk. Going to close, if you tell me it's still happeneing I'll reopen.

@ComfyFluffy
Copy link
Contributor Author

ComfyFluffy commented Nov 27, 2023

This still happens to me after updating wgpu (2964eed). Using macOS 14.1.1.

Sometimes resizing to fullscreen only will not trigger the issue. However, the issue will present swapping to other desktop using trackpad and then swapping back to the fullscreen app.

I have updated my git repo of the demo.

@cwfitzgerald cwfitzgerald reopened this Nov 27, 2023
@ComfyFluffy
Copy link
Contributor Author

ComfyFluffy commented Nov 27, 2023

image image

With further investigation using Instrument I found that VM: IOSurface has 9 GB in this run, which traces into CA::SurfaceUtil::CAIOSurfaceCreate(unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned long long, CA::SurfaceUtil::SurfaceAlignment, __CFString const*).

The stack trace in a single call:
image

@ComfyFluffy
Copy link
Contributor Author

ComfyFluffy commented Nov 27, 2023

With further investigation I located the issue to the execution time of surface.get_current_texture().

In the event loop, I use the following structure to render:

match event {
    Event::WindowEvent { event, .. } => match event {
        WindowEvent::RedrawRequested => {
            match state.render() {
                Ok(_) => {}
                Err(wgpu::SurfaceError::Lost) => {
                    warn!("Lost surface");
                    state.resize(state.size)
                }
                Err(wgpu::SurfaceError::OutOfMemory) => panic!("Out of memory"),
                Err(e) => eprintln!("{:?}", e),
            }
            println!(
                "Rendered in {}ms",
                render_start.elapsed().as_micros() as f32 / 1000.0
            );
        }
        _ => {}
    },
    Event::AboutToWait => {
        state.window().request_redraw();
    }
    _ => {}
}

Generally, state.render() takes about 8ms to finish (on 120hz monitor). Inside render() I use surface.get_current_texture() to get output texture. Normally it takes about 8ms to get the texture before rendering:
image

However, when putting the window into another desktop, or setting fullscreen mode then switch to other desktops, the window becomes fully invisible. This cause the time of calling surface.get_current_texture() changes to only a few microseconds:
image

Since the render has finished, Event::AboutToWait triggers, causing it to render again. This results too much frames being draw in a short period of time. Each get_current_texture() call allocates about 22M of memory, overwhelming the system memory.

@ComfyFluffy ComfyFluffy changed the title Memory Peak on Resizing to Fullscreen / Switching Desktops (macOS) Surface::get_current_texture does not block when the window is fully invisible (macOS) Nov 27, 2023
@Nnubes256
Copy link

Nnubes256 commented Mar 30, 2024

For those wondering how to mitigate this issue if affected by it while a proper fix lands, this comment I made for the previous big memory leak may help: sotrh/learn-wgpu#207 (comment).

The key thing of these two mitigations is avoiding drawing or limiting the number of frames drawn when the window is occluded, which helped with the previous issue; in this case, it might help in this case if you can detect whether the window is unfocused (thus, in particular, the CVDisplayLink hack I made may fix the issue entirely). Bevy mitigated this, for example, by limiting the rate of frames rendered when the app was in the background, see bevyengine/bevy#7611.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants