Faster PBR Material

[davehill00]

Is your feature request related to a problem? Please describe.

The PBR implementation in MeshStandardMaterial looks great, but is too expensive for devices like Oculus Quest/Quest 2. A scene that fills the entire screen with a quad using MeshStandardMaterial (with minimal features -- material color, a directional light, and an ambient light) renders at about 70Hz on Quest 2.

Describe the solution you'd like

I'd like to add a cheaper PBR implementation than MeshStandardMaterial -- perhaps something like this.

Describe alternatives you've considered

The alternative is using Phong material, but it's hard to translate PBR materials (e.g., from a GLTF file) into Phong materials.

Additional context

I'm planning to take a run at adding this kind of material to the ShaderLib/ShaderChunk system, but would certainly appreciate guidance or collaboration from folks who are more familiar with how this works. I'm still very new to the Three.JS codebase.

[liuyehua]

Good idea...

[Mugen87]

Related #14570.

[mrdoob]

/cc @elalish @WestLangley

[Ben-Mack]

Related: #19498

[arpu]

this is what hubs do https://github.com/mozilla/hubs/blob/master/src/utils/material-utils.js#L126
works good to translate the MeshStandardMaterial from glb/gltf to Phong Material

[elalish]

I'm definitely in favor of this. Not only could the Standard Material use some performance tuning, it could use a refactor/rewrite for code clarity. Still, this is likely to result in significant rendering differences, so I'd like to see this progress in parallel to the Standard Material and then when we get it to where it looks to be better, go ahead and replace the original. I work with the glTF standards group and I have tools to help evaluate render fidelity and I'd be happy to give feedback. And @Ben-Mack's comment is also a good point: my PMREM could use some optimization too. Still, please don't resort to SH or dropping HDR; that leads to pretty terrible results for general lighting. If anyone has good ideas for how to perf benchmark these shaders or figure out where the bottlenecks are, that would be much appreciated. We don't want to over-index on a single GPU.

[donmccurdy]

I'd like to see this progress in parallel to the Standard Material and then when we get it to where it looks to be better, go ahead and replace the original.

I'm not confident a material that is meaningfully cheaper to render (e.g. similar to Phong or Lambert) is ever going to look better than the existing MeshStandardMaterial. Do we have ideas for that kind of optimization?

For example, a version of MeshStandardMaterial that does per-vertex lighting (like MeshLambertMaterial) will be much cheaper than the current PBR material, and easier to convert correctly to/from materials loaded from glTF. But certainly that will never replace the current MeshStandardMaterial.

A goal of "able to render the full screen with this material, at full device FPS" is probably a good goal, for some combination of VR and mobile devices.

[RemusMar]

I'm not confident a material that is meaningfully cheaper to render (e.g. similar to Phong or Lambert) is ever going to look better than the existing MeshStandardMaterial. Do we have ideas for that kind of win/win optimization?

Maybe not better, but good enough:
https://necromanthus.com/Test/html5/Lara_mirror.html
Click on the stage to switch between Phong and PBR.
And good enough + faster is already a winner.

[elalish]

I was looking at the first link from @davehill00, which is still physically-based (not Phong or Lambert), but with new approximations made for FP16 and such. I think those are well worth considering. Still, it's worth checking first to see if we're computation-bound or cache-bound. And it may well be there is more perf gain on the table from optimizing PMREM fetch than from all these new lighting approximations. Hence it'd be nice to have some numbers first.

[davehill00]

A bit more context from my end -- on the Oculus Browser team, we've been getting questions about rendering perf from teams using ThreeJS for VR experiences. I'd like to have an easy "out of the box" answer to point people at that will get them off on a reasonable path to maintaining acceptable performance, and the MeshStandardMaterial doesn't feel like the right path to recommend.

I had proposed "cheaper PBR" because most teams also seem to be relying on GLTF (which largely seems to assume PBR). Perhaps the suggestions around converting PBR material settings to Phong settings is good enough. If not, I wasn't proposing making changes to MeshStandardMaterial that degrade visual quality, but rather providing a lower-quality option that is still PBR but less demanding.

@elalish -- I'll try and get some data on Quest later this evening, but I'm pretty sure it's computation bound not cache-bound (I think you mean texture cache/texture bandwidth?) in my case (since I'm not using PMREM or any textures in my test scene).

[elalish]

@elalish -- I'll try and get some data on Quest later this evening, but I'm pretty sure it's computation bound not cache-bound (I think you mean texture cache/texture bandwidth?) in my case (since I'm not using PMREM or any textures in my test scene).

For a realistic scene you'll probably want to include both IBL (PMREM) and material textures, at the very least baseColor, normalMap, and metallic-roughness. Without those you really aren't getting PBR at all and you could probably just convert to Phong as you say. But on the other hand, full PBR may not be needed for backgrounds and terrain, so a full screen quad may also be a bit extreme.

[davehill00]

@elalish Tried to do some investigation this evening, but my results are confusing so I'll need to do some more digging.

The boxing experience I have here: https://davehill00.github.io/box/dist uses MeshStandardMaterial on floor, "red" wall, bag, gloves, etc. (but not the white walls). There are no lights in the scene, but it's using a PMREM to light things and also has baked lightmaps. This runs at a consistent 90Hz. I added a directional and ambient light, and it still runs at 90Hz.

I also have a simple room (just an inverted box with the starting point inside) and it runs consistently below 70Hz on the same headset. I've tried simpler and more complex materials, adding a PMREM, etc. and it's always slower than the boxing experience.

Can't explain what I'm seeing, so need to keep digging. :\

[mrdoob]

@davehill00 do you have a link to the 70Hz room?

[davehill00]

@davehill00 do you have a link to the 70Hz room?

It's not up on GitHub yet. Will try to put something up tomorrow.

[davehill00]

@mrdoob here's the simple room scene that runs at ~68Hz on my Quest 2. Haven't had a chance to dig deeper into what's going on yet.
https://davehill00.github.io/template/dist/

[davehill00]

Oof... okay, this is embarrassing. :( I had set a background image (cube map, from a 32-bit EXR) on the scene, which was rendering behind the wall but not contributing anything to the final scene. So that's where all the extra cycles were going, apparently. A little surprised it was that expensive, but definite explains the discrepancy between the simple scene and the more complex boxing one.