Metal Rendering on macOS #1326

Does not yet link - async sockets missing implementation Migrating Metal-cpp over to CMake management Also rolling Metal-cpp up to 13.3 SDK.

I think it was set this way for debugging an issue, but the old code is causing issues itself.

Treating normalized co-ordinates as if they were full range. Broke some things with inverted alpha. Stuff like the side of the tent in city should render correct now.

Texture is being bound to the wrong index. This was causing the cleft to fail to load.

Eder Delin has had a 0 span since the resync, which Metal choked on

Should fix menus and partially fix linking books. Books still aren't drawing their contents, but that doesn't seem to be a renderer issue. The scripts that draw the books may not be working for some reason.

Was using setFragmentBuffer, not setFragmentBytes. Also was doing fast and then slow encoding back to back.

Also fixing several rendering bugs: - Single layer color materials now render correctly - Fixed lighting issues on moving surfaces - Fixed shader compilation crash on Intel HD Metal drivers by removing inline notation from shader functions. No performance decline was observed.

Intel graphics have unified memory, but don't support memoryless textures. Now explicitly checking for the Apple GPU family.

Thats a virtual interface, can't just rename that. Ooops!

Plasma is clearing the main drawable before it does offscreen drawing. In Metal, clearing and beginning drawing are linked. Need to cache the clear command until the main drawable pass begins. Strangely enough offscreen buffers like the shadow buffer still aren't clearing. This doesn't match what I've seen from the DirectX branch. Needs further investigation.

Avatar was self shadowing because the shadow map reflected the avatar state from the previous frame. The LUT for the shadow maps was a workaround to turn the x position into a color value. Removing the LUT in since we can just directly use that value in Metal instead of needing a texture go between. This also should slightly improve shadow quality over DX. Additional LUT removal

Ahnonay is loading now. But there are clear water rendering issues in Ahnonay that will need to be resolved.

Fixes directly to the WaveDecEnv shader. Minor changes to WaveSet7. Fixes for function hash table confusing different shaders and not tracking blend mode properly. Possibly related to GPU crashing, and fixes blend mode issues with shaders.

Water in Ahnonay uses it, but it wasn't quite enough to prevent all collisions

Metal was getting a pointer to a uint8_t when it expected a short for the UV count. This was leading to UV count sometimes being read wrong.

sqrt in HLSL 1.1 implies an absolute value taken of the value, according to the spec. This fixes an issue where water might not render from some angles. This bug could also be present in other shaders, I have not checked yet,

Some aspects of this are temporary until multiple wieght blending is introduced into the GPU pipeline. Integrated into the vertex shader. Not sure if this should be a compute shader eventually.

When I first wrote this I picked just a really small number. Now I understand it should be 1.f/255.f to represent the smallest test DirectX can do. Some areas - specifically the Barons Office - are very sensitive to this value. The view outside the windows has some layers where the alpha values should be eliminated - but they hover just barely above zero. A test too small won’t catch them. I’m not sure the purpose of these layers - they don’t seem to add anything visually - but they are there in DirectX too with the same lack of visual contribution..

The intro movie keeps its own run loop which prevents our autorelease pool from collecting memory until the intro movie is done. This leads to a pile up of gigs (or tens of gigs) of memory as no framebuffer can be released. Apple was nice enough to add NSAutoreleasePool to their list of bridged C++ classes though, so we can insert our own autorelease pool to capture that memory right into the pipeline.

State tracking is meant to reduce the number of redundant calls to the Metal command encoder. The DX renderer had something similar, but implemented differently. This is the beginning of improving state tracking. I’d like to move materials over to bindless rendering, which should also improve things. There are also more general redundancies to cut down.

This change has caused a performance decline. It’s likely related to the samplers being bound now instead of being hardcoded. Still investigating.

Metal didn’t like the dynamic array lookup I was doing. Converting it it to static lookups. This has restored expected performance. Also converted the sample type to a function constant. There is cleanup to do now. The fragment uniforms have been reduced to the alpha threshold, which is now awkwardly in a struct by itself. None of the layer data lives there anymore. In the pipeline, the encoding for shader lookup has gotten real messy. I’m probably going to look at adding MSAA first, but the cleanup is really overdue. I’ve had to work through multiple issues already that happened because I tried to do minor tweaks to that code but forgot to touch a specific part.

Using the power of C++ to move the sampleLayer function into the layers struct. The function can now directly access struct members and doesn’t need them passed as arguments. Having trouble with the array lookup of samplers on Intel graphics and hoping this helps.

Was using the wrong samplers

Adding a clear shader to clear depth/color as needed without staerting a new pass

Shaded material states like water weren’t correctly hashing and testing equality based on span properties

MSAA textures are now memoryless. Vertex shader has been improved to reduce memory bandwidth and encoding time.

Also getting rid of the buffers and arguments used to back it. They were only used for lighting, which is in the vertex pass.

Ki screenshotting no longer means death

Texture needs to be blitted out to CPU for discrete cards.

After a resolution change cubic render targets are always marked as dirty. This causes problems with the KI snapshot renders which will try to recreate the buffer and not re-render it.

This was causing a Metal compiler crash because inconsistant function constants were being passed.

We need to supply at least one display mode. This isn’t perfect - it causes graphics options to set the plate/cursor layer to 800x600. But it’s better.

My optimization broke shadow cast rendering. :( Fixed!

The new manual clear could have its draw culled if backface or frontface culling was set before the clear pass. Disabling culling. I could wind my rect the other way - but the engine is allowed to front face cull. Easiest to just turn it off.

…th the sampleTypes being defined as size_t - which is way too large anyway. uint32_t is probably too large too but going with what works for now. Also doing some other cleanup and fixing unified vs Apple family capability mixups.

That should be LoadActionDontCare, and StoreActionStore. I had the opposite. Whoops!

Bitten by this again. There’s got to be a better way. Also dropping the size down to uint8_t.

This allows the intro movie to properly resize (which relies on the plate manager getting re-created.)

That wasn’t supposed to be in the last commit

This should align the framebuffer device and the render device on dual GPU Intel Macbook Pros. Previously it would try to draw to an Intel IGP framebuffer while rendering on the discrete card. The way this is implemented is a bit of a hack. The Mac client app probably needs a deeper refactoring.

The water vertex shader needs to square as part of it’s reference math, but the value it needs to square can sometimes be negative. HLSL usually just does the “right” thing but MSL (and GLSL) need more direction and can treat squaring or square rooting a negative as undefined behavior. Absolute valuing first - in since we know that a square will always produce a positive. Long term - this HLSL code was ported over directly. It’s possible that this code could be replaced by some other Metal functions like reflect. More research is required.

This was suplimented by the samplers that are set as pipeline resources to support configurable sampling.

Also adding run loop to take care of intro movie not running on main thread, and honoring full screen setting

The previous fog algorithm was very wrong, but also covering for a problem where fog was being applied to layers it should not have been

Flat color base layers were blending against themselves

A cube map near or in the vault in Kadish has inconsistant cubic mipmap face sizes, and will cause Metal to crash. This change will cause Metal to manually compute mipmap sizes, and then load the data anyway. This is what DX does. The mipmaps affected don’t seem to be intended to be used.

Calc lighting was done before projections were applied. This was allowing fog (which is part of lighting calc in Metal) to be enabled for projection layers. This was most visible in Kadish. This code is kind of messy, but has been through several improvement passes already.

This fixes steam in Gira and probably other things. This function is still kind of a mess and needs more cleanup. The DX version is also messy though.

Have to pop the layers. Code is still messy and needs a better solution.

Caching z bias state. Also caching cull mode state.

Should fix bugs with water not drawing and cull mode not being tracked properly

Clearing up redundant loops, clarifying code, fixing some latent possible issues with pushover layers not being consistantly applied

Still has known issues to fix

Reflections were having shadows renderer on them with out of range shadow values. This clamps the values so they can’t be out of range. The DX implementation used a LUT texture here that implicitly clamped the range.

Fixing case where age UV index was encoded out of range (Great Tree Pub), and texture had no usable mipmap levels (GoMePubNew)

Textures can now be stored in private storage in Metal - which means on discrete cards they can be VRAM only. This feature is disabled on Apple Silicon - but perhaps should be enabled. Even though Apple Silicon doesn’t have VRAM, the GPU might be able to store a private texture in a better storage format.

Description is a string, not the texture description.

This is a leftover from the initial Metal prototype

Ranges is a C++20 feature I didn’t end up using anyway

Also forcing plMetalTextFont and plates back through the devices pipeline state management to match the current color space

Shadow blurring would be the best improvemnt for now (along with upgrading shadow resolution.) This could come back in the future - but for now it’s not really helping much.

All textures now get blitted to private memory.

Lights can now go into a variable length buffer - trying to reduce memory traffic. Ideally the pipeline should probably hold these as a vector, but this is a start.

Also fixing finding client data when app is launched from Finder

The engine doesn’t know the size, so it keeps setting it to the wrong one. The default viewport size is just fine anyway.

I thought I already fixed this?

Copy depth is 1, not 0

This removes the function constants for the sampler type - and all the logic that goes with. Samplers are now encoded in an array. This should simplify what the fragment shader has to do - and should fix Intel Integrated Graphics which was unhappy with samplers as a non-const variable.

Sampler state code is now shared. I’m not restoring the monolithic “encode all the layers” treatment that DirectX did. I need some flexbility to encode argument buffers at a later point, which means the material can’t just bind itself to encoder states inside of a black box function.

Cleaning up the variable names as well. Spot lighting was broken on Intel Integrated Graphics. The spot lighting intensity wasn’t clamped before the falloff calculation.

Position should be calculated only once now in the vertex shader. Position will also be blended with any skinning matrices before the lighting pass. That’s technically a fix - even though I haven’t seen any defects because of that yet.

Kadish light spots on ground now project (seemingly) correctly. Lighting was not correctly configured.

Shadow light needs to be zeroed

Avatar textures for other players might be loaded in after material shader creation. That means the material shader state needs to be recreated for a color -> texture transition. Setting the dirty flag when an avatar texture is set - and checking it in the material ref.

The lighting struct was being written wrong when projection lights are also present. There’s a bit in lighting where it scales down lights that aren’t rendered which - makes no sense.

Restoring the original light count for Plasma for Metal. 32 lights is a lot - but there are a lot of visible improvements. This should possibly be tiered based on hardware.

This should fix Ahnonay in it’s current state on Trollland

Cubic bindings will no longer be present for layers that don’t have cubic textures.

Need to reset the index buffer earlier in the render span function. Render all projections was a late addition, and it got added to the wrong place relative to the buffer restoration.

Shadow commands were being given inconsistant function constants on compliation. The shader is looking for the source types to be properly encoded as constants, but the shadow shader was hard coded to look for the source type constant for layer 3. This caused layers 1 and 2 to be dropped and not properly bound because the shader assumed there was nothing there.

Streaming of subsets of vertex indexes was broken, fixing. Because of triple buffering or the index buffers, there might still be bugs. A partial update of a buffer should be applied on the previous version of the buffer, but we have to buffer swap. Should consider copying the previous buffers contents into the new buffer before updating for consistancy. WaveRip was also badly broken. Fixing.

Haven’t added platform specific scoping yet

Co-authored-by: dgelessus <[email protected]>

Only OpenAL soft support now

Shadow stages 0 and 1 and the normal stages 0 and 1 conflict and were tromping over each other. Redoing this part of the pipeline to allow for alpha blend textures to be sourced from textures 0 or 1, while allowing the alpha blend texture coordinates to be derived in stage 3.

Swapping the shaders around to support row major hsMatrices and eliminating the swap function. This change hasn’t filtered down to the dynamic effects which get their uniforms through a giant buffer provided by the engine itself.

Moving to using hsGMatStateEnums.h

Old sampler bindings were still in place hiding that per layer sampler binding was taking place during shader pre-compile. That caused state confusion because render encoders aren’t available during pre-compile. Removing old sampler bindings and making sure sampler binding only happens during draw - not precompiles.

Shadows no longer need to alter light state - which is prep for the light refactor. Adds a programmable shadow pipeline that has it’s own state independent of the light system. Rendering of shadows is now done directly instead of through light manipulation and the “fixed” function vertex shader.

…layer 0 without updating state tracking. (Perhaps sampler state changes should be wrapped in a function.)

Should allow tests to build on Mac again. pfFeatureInc included the Metal pipeline header, but didn’t declare it as a dependency.

Or rather the lack of UBO encoding.

I likely have not found all the C style casts.

…venting a delegate message from properly setting Plasma resolution on launch.

…e callback (cherry picked from commit bcb7266f29fc71b136ce9c973fe41aac445ebfec)

The render command encoder is create lazily - and clear is not a draw call in Metal. A clear only pass meant no render was being encoded for the command buffer. The render command encoder management could use a bit of cleaning - but for now manually forcing the lazy creation of a command encoder. This could be probably moved to a non-lazy model - but it’s lazy right now because we’re holding for a clear command.

Commits on Nov 27, 2023

Review feedback updates

colincornaby committed Nov 27, 2023

Configuration menu

View commit details

Copy full SHA for 6c96be0

Browse repository at this point

Copy the full SHA

6c96be0 View commit details

Browse the repository at this point in the history

Commits on Nov 28, 2023

Removing clang format file

colincornaby committed Nov 28, 2023

Configuration menu

View commit details

Copy full SHA for 245b00f

Browse repository at this point

Copy the full SHA

245b00f View commit details

Browse the repository at this point in the history

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Metal Rendering on macOS #1326

Metal Rendering on macOS #1326

Commits on Nov 14, 2023

Commits on Nov 16, 2023

Commits on Nov 18, 2023

Commits on Nov 25, 2023

Commits on Nov 27, 2023

Commits on Nov 28, 2023