Exposing Mesh Merging Functionality

[lawnjelly]

Describe the project you are working on

Nearly all 3D games.

Describe the problem or limitation you are having in your project

Performance bottlenecks due to drawcalls, state changes, and node overhead.
Also relevant: #182

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Although mesh merging functionality was introduced in Rooms & Portals, and exposed to binding in godotengine/godot#57661, it would be nice to expose this in a user friendly manner and make it easy to use and fit as many use cases as possible.

This is especially relevant for Godot 3.x, as OpenGLES is particularly sensitive to drawcalls and state changes, but would also be beneficial for 4.x, especially the GLES3 backend.

Note that although the merging backend functionality is currently purposefully very simple and limited, it can easily be improved, so this should be considered separately from the topic of user interface, which is the subject here.

Shadow proxies

As well as merging meshes in terms of display with materials, merging meshes is also relevant for the creation of shadow proxies, because the stringent requirements for material similarity can be relaxed. Essentially almost any group of opaque meshes transformed as a group can be combined into a single shadow proxy, and greatly reduce drawcalls.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

There are 4 obvious places we can perform mesh merging:

• At import time
• As a pre-process in the editor
• At export time
• At runtime when loading / running the game

All these have pros and cons, and there is no reason why we cannot implement merging at more than one stage.

Currently I'm of the opinion that at a minimum, we should offer merging at runtime, as that covers all use cases, especially procedural. I would also be in favour of also offering merging at one of the earlier stages, primarily because of the benefits in terms of level load time, as this can be a pre-process.

Import merging

pros

• This could be useful for large models that are built in e.g. blender that will run inefficiently as is in the engine. Could ensure that the format of the model in engine is kept simple.
• Can take as long as required

cons

• This can also be done with all the other options, and the other options more versatility.
• The merging is only occurring at the level of single objects. This makes it only relevant in a subset of cases, where a complex scene / model has been exported. When you place several of these objects in a scene later, they will not be merged.

In editor

Example draft PR:
godotengine/godot#61564

pros

• Very versatile and allows the user to neatly structure their scene graph, and visualize what will occur at runtime.
• Can take as long as required

cons

• Tends to be a one way process, unless the user keeps the old nodes / source scene.
• Probably would require manual management of shadow proxies (unless combined with a later approach)

At export

pros

• Minimizes chance of merge bugs at runtime
• Does not require one way changes during editing
• Can take as long as required

cons

• Requires merging "groups" to be identified ahead of time.
• Unsuitable for procedural content
• May be complex to implement

At runtime

Example draft PR:
godotengine/godot#61568

pros

• Most powerful approach, can be tailored to every situation
• Works with procedural content

cons

• Can slow down level load
• Due to the need for performance, time consuming merging techniques may not be practical

If this enhancement will not be used often, can it be worked around with a few lines of script?

Although mesh merging can be done entirely from gdscript, or using the existing MeshInstance::merge_meshes() function in 3.x, we want to make it as easy to use as possible (and port to 4.x).

Is there a reason why this should be core and not an add-on in the asset library?

Depending on the implementation, may require being core for runtime performance. Could possibly be implemented as gdnative in 4.x but most seem keen on this being available in core.

Discussion

• There are a lot of potential tie-ins with hierarchical level of detail (HLOD), so we should consider that especially for 4.x (HLOD has not yet been implemented in 3.x).

[Zireael07]

My vote is for runtime at first. After all, portals processing ALSO slows down level load a bit, and no one seems to be complaining about that.

I'm not that much set about having the functionality available in editor, but (again a parallel to portals) groups would potentially let us have runtime merging with some sort of a preview in editor so that you know what you're merging.

Could possibly be implemented as gdnative in 4.x but most seem keen on this being available in core.

That's because competitor engines offer similar functionalities (Unity's static meshes, UE4 actor merging) out of the box.

[clayjohn]

I'm putting this on the list for proposals review, I would like to discuss synchronously as we need to discuss in the context of how many features we are comfortable adding to 3.x that aren't in 4.0. As this starts to significantly overlap with 4.0's HLODs we need to make sure that we implement this with a clear plan about whether it will be added to 4.0 and if so, what that will look like.

CC @JFonS Your input would be very helpful here as you are the current expert in HLOD

[lawnjelly]

I'm putting this on the list for proposals review, I would like to discuss synchronously as we need to discuss in the context of how many features we are comfortable adding to 3.x that aren't in 4.0. As this starts to significantly overlap with 4.0's HLODs we need to make sure that we implement this with a clear plan about whether it will be added to 4.0 and if so, what that will look like.

Agree here, I'm of the opinion we can probably use the same approach here for 3.x and 4.x (and this proposal is addressed at both), but some feedback regarding playing nice with the existing HLOD would be useful.

On a "code arrangement" front - now I've been test writing some shadow proxy merging code, I'm wondering whether we should consider the pros and cons of splitting the merging functionality out into a helper class (or even putting in geometry or some existing helper class) rather than putting it all in MeshInstance. In some ways it would be nice to keep MeshInstance as lean as possible imo so it is easy to understand, and doesn't become a class that attempts to do too many things. But equally I'm aware that sometimes we prefer to keep localized solutions.

[lawnjelly]

Just to note discussing this on rocket chat regarding Godot 4:

reduz
lawnjelly merging meshes on master makes more sense at import time
Since we have the import UI

lawnjelly
I wrote a proposal on this in fact: godot-proposals#4630
On the pros cons of merging at various points.
Merging at import can be useful in some cases, but it doesn't help merge objects in the scene.
It only can help when you are importing complex objects.

reduz
lawnjelly the problem of doing it in the scene is more that workflow wise its kind of wasty
And you dont get auto updates if something changes

lawnjelly
Ah that is discussed in the proposal.
There are 4 points for merging covered there, and only merging in the editor scene has that problem.

reduz
In Unity this may make sense because they post process the scene on export
In Godot 4 I think auto batching will make a bit more sense using instancing
As state changes are a LOT less expensive

lawnjelly
Yes, I think the benefits are less with vulkan, as it is more useful for reducing drawcalls in GLES.
But it may still be useful for the GLES3 backend in 4.0.

reduz
We should likely use the same approach of auto batching in the GLES3 backend of 4.0
Not as cheap but should be good enough

lawnjelly
Instancing also relies on objects being the same type though?
What happens if you have a bunch of objects of different types (different geometry) but using the same materials?

reduz
Yeah
That is not as cheap in GL as in Vulkan
But its still generally good

lawnjelly
We can probably wait and see for 4.x, or do some tests.
It's pretty easy to do either way.

reduz
Alright!
Worst case it can also be done as extension in 4.x

lawnjelly
Yes an extension is something I wondered about for 4, as it is more easily done, for people who want it.
The extensions is quite exciting for moving a lot of stuff into them rather than core.

This sounds pretty reasonable. The main benefit is likely to be in Godot 3.x, and the GLES3 backend in Godot 4, and it should be quite possible to do this as an extension in 4.x. 👍

So for now I am happy to continue experimenting with the PRs for 3.x (as the benefit is quite significant there), and will have a look at the extensions in 4.x a little further down the line (it doesn't sound like a "must have" for 4.0 release).

[clayjohn]

Discussed at Proposal review, next steps:

1. investigate being able to toggle merging at export (or consider baking in the editor)
2. Investigate if this can work well with HLODs in 4.0

[lawnjelly]

Based on our talks in the proposal review, I've done some timings for the mesh merging at runtime, turns out my estimate of 2 seconds for trucktown was wildly out 😀 . It was the timing for 200,000 boxes I used for optimization I was thinking of.

Truck town : 4 milliseconds
WroughtFlesh : (merging "entire world", the main level which is quite large) 214 ms
200,000 boxes : 2160 ms
20,000 boxes : 204 ms

This is actually not too bad, even considering that merging on e.g. a mobile is likely to be slower than on my low end PC. Also I suspect the cost of merging is likely to be non-linear, it is probably more likely to depend on the number of meshes (which the 200,000 boxes test case stresses), rather than the vertex count. There is nothing very computationally expensive going on there.

I'll be looking at whether we can do some of this during the export (as this will be faster), but it is looking like doing it all at level load is a very viable option. Another thing which occurred to me - if we merge at export, this will mean the pck file will likely be larger, as most maps will contain duplicated objects, so this is another plus for doing it at runtime.

Performance improvements

Incidentally, although these are mostly on the PRs themselves, to give a very approximate idea of FPS improvements:

• Trucktown (using small screen and shadow map) - 1.5x
• WroughtFlesh : 2x
• 20,000 boxes : 24x
• 50,000 boxes with directional light (and shadow proxy) : 46x

(The differences of course depend on what part of a level you are viewing, viewing angle etc. In general we should aim especially to make the worst cases better, as that is generally the critical factor.)

The improvements (or not) due to merging are totally dependent on how bottlenecked you are by drawcalls and state changes (and node overhead). Trucktown for instance doesn't have a huge amount of objects, and is bottlenecked by shadows / fill rate, so you only tend to see FPS improvement with merging when you have a smaller screen size and small shadow map (to shift the bottleneck).

Wrought Flesh as a full game (rather than simple demo) has lots of different objects of different types and benefits more from merging.

The 20,000 boxes benefits the most because it is limited by drawcalls etc. This is likely to be the situation in open world type games.

Of course this all has to be balanced against the drop in culling accuracy.

[JFonS]

I think mesh merging and HLOD are two separate concerns without much overlap.

Mesh merging should be used in cases where the benefit of having a single draw-call outweighs the opportunities for frustum/occlusion culling. I am thinking of cases where the same "object" is composed of many smaller items and most of the time you either see them all or you see none of them (i.e. a cabinet full of smaller elements, a small room with its furniture).

On the other hand, HLOD is meant to be used in cases where frustum/occlusion culling makes sense up-close, but you want to reduce the draw-call and polygon count at a certain distance (i.e. a group of buildings or a forest being turned into a single mesh when far from the camera).

So the only overlap I can find between those two use-cases is using the mesh merger to set up the lower LODs. For example, users could place a bunch ow low-poly trees under a MeshMerger node that would then be used as an impostor to replace a bunch of high-detail trees. This use-case should not require any special changes. As long as the MeshMerger node is a GeometryInstance, it can be part of a visibility dependency tree.

[me2beats]

say I have a donut, but it uses 3 meshes, so I can't use it as is for GridMap or MultimeshInstance.
Could the proposed mesh merging help in this case?

[lawnjelly]

say I have a donut, but it uses 3 meshes, so I can't use it as is for GridMap or MultimeshInstance. Could the proposed mesh merging help in this case?

I'm not super familiar with either, but yes, if they demand a single MeshInstance. The mesh merging I'm also intending to be able to optionally combine MeshInstances with different materials so that these form different surfaces of the same new MeshInstance. (Sometimes you'll want this, and sometimes not, because when combined they are culled together.)

I'm not absolutely sure whether MultimeshInstance or GridMap demand a single surface MeshInstance or whether they work with multiple surfaces (materials).

EDIT: Yes, this does work. 👍
Tested with the latest MergeGroup PR. A good workflow is to bake the MergeGroup, then save the merged mesh as a .mesh file, which can then be loaded in the MultiMesh. The key to getting the multi materials to work is to set them in the mesh surfaces, rather than mesh instance surfaces.

[Radivarig]

MergeGroup was just released in Godot 3.6. Can it be ported easily to 4.x?

[Calinou]

Can it be ported easily to 4.x?

It's technically feasible (but is quite a lot of work). However, right now, we're waiting to see if the 3.6 mesh merging implementation sees enough use in projects to make it worthwhile to port to 4.x. If it turns out to be worth it, this will also be an opportunity to revise the exposed API for mesh merging according to feedback.

I also expect performance benefits to be less significant in 4.x due to automatic instancing in the RenderingDevice-based rendering methods, and the fact that draw calls are generally cheaper in Vulkan/Direct 3D 12/Metal compared to OpenGL.