macOS Metal: validation layer error about MTKView texture lifetime when app window is obscured

[floooh]

Repro:

• on macOS 13.01, Xcode 14.2
• run any of the sokol-samples with the Metal backend in the Xcode debugger, make sure the Metal validation layer is enabled
• put the Xcode window on top of the sample window so that the sample window is completely obscured

After a little while (sometimes under 1 second, sometimes longer) the sample crashes inside [MTKView currentRenderPassDescriptor] (called from within _sg_mtl_begin_pass) with a Metal validation layer error (see below).

More details:

• happens with or without ARC enabled
• it does not happen in the Xcode game sample project
• happens both with SG_NUM_INFLIGHT_FRAMES = 2 or 3 (the Xcode sample uses 3 inflight frames)
• maybe related to this ticket? macOS/Metal: currentDrawable can return null, freezing the render loop. #504
• also happens with Xcode 14.01 and 14.1 (although it took a very long time 10..20 seconds?) before the problem triggered - getting my hopes up just to be crushed :D
• also happens with sample_count == 1

-[MTLDebugDevice notifyExternalReferencesNonZeroOnDealloc:]:2885: failed assertion `The following Metal object is being destroyed while still required to be alive by the command buffer 0x138008200 (label: <no label set>):
<MTLToolsObject: 0x106f04f60> -> <AGXG13XFamilyTexture: 0x106f04b40>
    label = CAMetalLayer Display Drawable 
    textureType = MTLTextureType2D 
    pixelFormat = MTLPixelFormatBGRA8Unorm 
    width = 800 
    height = 600 
    depth = 1 
    arrayLength = 1 
    mipmapLevelCount = 1 
    sampleCount = 1 
    cpuCacheMode = MTLCPUCacheModeDefaultCache 
    storageMode = MTLStorageModeManaged 
    hazardTrackingMode = MTLHazardTrackingModeTracked 
    resourceOptions = MTLResourceCPUCacheModeDefaultCache MTLResourceStorageModeManaged MTLResourceHazardTrackingModeTracked  
    usage = MTLTextureUsageRenderTarget 
    shareable = 0 
    framebufferOnly = 1 
    purgeableState = MTLPurgeableStateNonVolatile 
    swizzle = [MTLTextureSwizzleRed, MTLTextureSwizzleGreen, MTLTextureSwizzleBlue, MTLTextureSwizzleAlpha] 
    isCompressed = 1 
    parentTexture = <null> 
    parentRelativeLevel = 0 
    parentRelativeSlice = 0 
    buffer = <null> 
    bufferOffset = 0 
    bufferBytesPerRow = 0 
    iosurface = 0x6000000180f0 
    iosurfacePlane = 0 
    allowGPUOptimizedContents = YES'
-[MTLDebugDevice notifyExternalReferencesNonZeroOnDealloc:]:2885: failed assertion `The following Metal object is being destroyed while still required to be alive by the command buffer 0x138008200 (label: <no label set>):
<MTLToolsObject: 0x106f04f60> -> <AGXG13XFamilyTexture: 0x106f04b40>
    label = CAMetalLayer Display Drawable 
    textureType = MTLTextureType2D 
    pixelFormat = MTLPixelFormatBGRA8Unorm 
    width = 800 
    height = 600 
    depth = 1 
    arrayLength = 1 
    mipmapLevelCount = 1 
    sampleCount = 1 
    cpuCacheMode = MTLCPUCacheModeDefaultCache 
    storageMode = MTLStorageModeManaged 
    hazardTrackingMode = MTLHazardTrackingModeTracked 
    resourceOptions = MTLResourceCPUCacheModeDefaultCache MTLResourceStorageModeManaged MTLResourceHazardTrackingModeTracked  
    usage = MTLTextureUsageRenderTarget 
    shareable = 0 
    framebufferOnly = 1 
    purgeableState = MTLPurgeableStateNonVolatile 
    swizzle = [MTLTextureSwizzleRed, MTLTextureSwizzleGreen, MTLTextureSwizzleBlue, MTLTextureSwizzleAlpha] 
    isCompressed = 1 
    parentTexture = <null> 
    parentRelativeLevel = 0 
    parentRelativeSlice = 0 
    buffer = <null> 
    bufferOffset = 0 
    bufferBytesPerRow = 0 
    iosurface = 0x6000000180f0 
    iosurfacePlane = 0 
    allowGPUOptimizedContents = YES'

[floooh]

...it works when using the vanilla [MTLCommandQueue commandBuffer] method to create the frame's command buffer instead of [MTLCommandQueue commandBufferWithUnretainedReferences].

Maybe MTKView expects that a command buffer with 'retained references' is used, and otherwise the lifetime management gets messed up? Does MTKView have a configurable 'pipeline depth'?

[floooh]

...in any case, sokol_gfx.h should probably have a config flag for whether commandBuffer or commandBufferWithUnretainedReferences is used, because it looks like this decision 'leaks' into the window system glue.

[floooh]

Random thought: maybe use two command buffers? One with unretained references for 'regular rendering', and one with retained references for commands that use swapchain resources.That way the swapchain textures would be kept alive for as long as needed, but regular commands wouldn't have the refcounting overhead...

[floooh]

Some things to try out via Twitter (https://twitter.com/Alecazam123/status/1603445616234745856)

I’d recommend setting the drawables to write only if you aren’t already. brga is good since it avoids a copy in the old days. Any query for the drawable descriptor or drawable is a pacing stall. Never use 2 drawables, since the display link is a terrible compositor api.

[floooh]

Ok, sokol_gfx.h now uses a second MTLCommandBuffer created with retained references which only holds the presentDrawable command and which is scheduled after the regular command buffer (which still uses unretained references). This appears to fix the problem.

Fix is in commit: 9cba653