KHR_blend

[donmccurdy]

Complement to #1296, in the hope that blending can be kept WebGL-agnostic. DirectX supports many blend operations that GL does not; currently those are omitted from this extension for compatibility. I also did not attempt to include gl.blendColor.

BabylonJS and Unreal appear to skip many of these blending equations. I wouldn't be opposed to cutting back to just the intersection, but would like to get more feedback on this first. @bghgary any preference, or rationale behind Babylon's choices here? Found this thread.

TODO:

• Clarify alphaMode == BLEND requirement. Is extension invalid when this is not the case, or simply ignored? Do we allow alphaMode === MASK, too?
• Clarify that MIN and MAX equations do not use blendFactors.
• Consider simplifying allowed blend states.
• Replace GL integer constants with string enums.

References:

• Unity: Blending
• Unreal: Blend modes
• OpenGL: Blending
• THREE: Blending equation constants
• BabylonJS: Blend modes
• WebGL constants
• Visual glBlendEquation tool

Reference: https://user-images.githubusercontent.com/1848368/38159786-bca1e3ea-3464-11e8-9d3b-9d1a1e8cac74.jpg

[deltakosh]

For Babylon.js, we actually exposed only required and used ones. Not a big deal to add more but no user ask for them so far

[donmccurdy]

Thanks, good to know that it hasn't been a common request, but that implementing more is an option.

I'm open to trimming this down; my requirements are basically just additive blending right now. But as-written the extension isn't that complex, so writing additional extensions later for additional blending modes feels a bit silly. Would welcome more opinions here.

/cc @stevesan

[stevesan]

Thanks, Don. Additive is definitely the big one, but multiplicative can be useful too for darkening things in an order-independent way. For example, for Blocks glass rendering (example: https://poly.google.com/view/atB26Z6BPd0), we use both a multiply pass and an additive pass.

[recp]

For multiple lights, what will happen in this scenario if EXT_blend exists as different than ADDITIVE?

Multi-pass lighting (forward rendering):

for light in lights {
  /* additive blending for other lights */
  if light != firstLight {
      glDepthFunc(GL_EQUAL);
      glEnable(GL_BLEND);
      glBlendEq(GL_FUNC_ADD);
      glBlendFunc(GL_ONE, GL_ONE);
  }
  
  renderForLight(light);
}

if material needs multi-pass then the blend equations would make sense here:

for light in lights {
  /* Step 1: render material to FrameBuffer */
  for pass in material->passes {
     if pass != firstPass {
        /* set blend options specified in `EXT_blend`extension */
     }
  }

  /* Step 2: use/bind FrameBuffer texture as material (IF IT IS POSSIBLE FOR EVERY CASE) */

  /* Step 3: Additive Blending for other lights */
  if light != firstLight {
      glDepthFunc(GL_EQUAL);
      glEnable(GL_BLEND);
      glBlendEq(GL_FUNC_ADD);
      glBlendFunc(GL_ONE, GL_ONE);
  }

  renderForLight(light);
}

otherwise I don't know how to render EXT_blend-enabled glTF model :( Thanks for implementation tips if it is possible in multi-pass lighting scenario.

Also some Order-Independed Transparency techniques e.g. Weighted-Blended OIT requires additional blending operations it may cause blending-conflicts too

[MiiBond]

@recp I think you may need to separate transparent passes from your multi-pass lighting. If I remember correctly, UE3 (which did multi-pass lighting as you describe) didn't allow dynamic lighting on transparent objects for this reason.

Perhaps you could render transparent objects in a single pass with a clamped number of lights? Or, require baked lighting for transparent objects? I'm sure there are better options too.

These are implementation details and they really depend on how your runtime is handling its rendering. I'm not sure that the spec of KHR_blend can specify how to implement it in every case.

[recp]

@MiiBond thanks for your advices,

Actually I render scene in three passes if transparency is active and it is working, it uses Weighted, Blended OIT (for now):

• Opaque Pass: Render opaque objects as above blending equations
• Transp Pass: Render transparent objects using different blending equations (multiple render targets)
• Composite Pass: Composite opaque and transparency passes using different blending equations

Full implementation: https://github.com/recp/gk/blob/master/src/transp/builtin/oit/weighted_blended.c#L91

I'll update lighting design after implementing morph/skin animations to support custom blending options in materials, then I can support this extension.

Also will this extension be valid for opaque objects too (alphaMode != BLEND)? If opaque means that pixels/fragments behind opaque surfaces are completely invisible then why do we need to specify blending for that surfaces? Maybe it should only be valid if alphaMode == BLEND?

[donmccurdy]

Thanks @MiiBond and @recp!

Maybe it should only be valid if alphaMode == BLEND?

I agree.

[MiiBond]

@donmccurdy I don't see FUNC_MULTIPLY as an option. Was that intentional? I think it's pretty important for modeling things like light absorption.

[donmccurdy]

So far this extension uses the equation and factor basics of OpenGL, which are combined for different effects. For example multiplicative blending would be:

"materials": [
    {
        "name": "MultiplicativeMaterial",
        "alphaMode": "BLEND",
        "extensions": {
            "EXT_blend": {
                "blendEquation": [32774, 32774], // ADD
                "blendFactors": [774, 0, 772, 0] // DST_COLOR ZERO DST_ALPHA ZERO
            }
        }
    }
]

Common blend types mentioned in Unity docs:

Blend SrcAlpha OneMinusSrcAlpha // Traditional transparency
Blend One OneMinusSrcAlpha // Premultiplied transparency
Blend One One // Additive
Blend OneMinusDstColor One // Soft Additive
Blend DstColor Zero // Multiplicative
Blend DstColor SrcColor // 2x Multiplicative

^All assume the blend equation is "add". Probably more of these details should be in the extension spec. 🙂

[recp]

Clarify alphaMode == BLEND requirement. Is extension invalid when this is not the case, or simply ignored?

Maybe if this extension is enabled we could have some options:

1. If alphaMode not exists AND EXT_Blend exists AND this extension is supported then alphaMode's default mode is assumed to BLEND instead of OPAQUE?
2. Both alphaMode and EXT_Blend exist AND alphaMode == OPAQUE then EXT_Blend could be ignored?
3. alphaMode == BLEND is a must for EXT_Blend

Having 1 and 2 could be an option, or just 3.

I couldn't see FUNC_MULTPLY in GL reference: https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBlendEquation.xhtml

Is FUNC_MULTIPLY DirectX feature? I don't know about DirectX, but in OpenGL if know correctly multiplicative blending (if you meant this) is glBlendFunc(GL_DST_COLOR, GL_ZERO) as described here: http://www.learnopengles.com/tag/additive-blending/

[MiiBond]

@donmccurdy Yeah, I need to have more coffee this morning. It's been a while since I've looked at blend equations. For some reason, I remembered there being an explicit multiply.

@recp I'm not sure that we'd want the presence of an extension to change the default on another property. I think we should just require the author to set the alphaMode value correctly.

[donmccurdy]

To fill in details for any future readers, the core equation is:

output = (<sourceFactor> * sourceFragment) <operator> (<destinationFactor> * destinationFragment)

Where:

• <operator> is determined by blendEquation
• <sourceFactor> and <destinationFactor> are determined by blendFactors
• sourceFragment and destinationFragment are current fragment shader output color and existing color in the frame buffer respectively.

So in the multiplicative example above, we get this, which simply multiplies the two fragment colors:

output = DST_COLOR * sourceFragment + 0 * destinationFragment

This article was helpful for me, I'm having to read up on this as well. 😅

[recp]

What about glBlendFuncSeparate? Could this extension also support it?

It seems WebGL also supports it: https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/blendFuncSeparate

Since EXT_Blend allows custom blending equations it could also support glBlendFuncSeparate too for specifying different blending for alpha. I'm not sure if it would make the spec (or ext) more complex. On the implementation side, it is just another function with different parameters, after implemented EXT_Blend 🤔

[donmccurdy]

@recp it is already covered — because blendFactors requires 4 values, it is implicitly always using glBlendFuncSeparate. I'm wondering if there is a performance difference in that, or whether blendFactors and blendEquation should also allow 2 and 1 values respectively.

[recp]

@donmccurdy thanks for clarification, I was thought that it is glBlendFunc, it should be obvious from params

On CPU side glBlendFunc may be faster (and less instructions) because RGB and A has same value, and it can be done with SIMD instructions e.g. SSE2. On GPU side, I don't know.

But my render engine's state manager can replace glBlendFuncSeparate with glBlendFunc if both RGB and A blending options are same AND previous blend state is glBlendFunc not glBlendFuncSeparate. I like this idea and added to my TODOs. 🤗

[ggetz]

Hi @donmccurdy and all, we have plans to support this extension in gltf-pipeline (PR CesiumGS/gltf-pipeline#387 (comment)). Are there any more updates expected here?

[ggetz]

Additionally, is EXT_blend the final name of this proposed extension, or will it be KHR_ or a vendor prefix?

[donmccurdy]

Renamed to KHR_blend.

The more likely updates to this extension are those mentioned in #1302 (comment). Most importantly, whether we should move away from the flexible but low-level approach here, toward just allowing a few common blend states. For example, only define ADDITIVE and MULTIPLICATIVE (or MODULATE). Those modes (and many others) can be composed from the low-level blend functions defined here.

Unreal only supports those two blend modes (in addition to the three already in glTF: opaque, masked, translucent), unless there is a mechanism to use more modes than Unreal docs describe here, so that's an argument for simplifying. On the other hand, Unity's docs claim the following modes are "the most common blend types":

Blend SrcAlpha OneMinusSrcAlpha // Traditional transparency
Blend One OneMinusSrcAlpha // Premultiplied transparency
Blend One One // Additive
Blend OneMinusDstColor One // Soft Additive
Blend DstColor Zero // Multiplicative
Blend DstColor SrcColor // 2x Multiplicative

[ggetz]

Thanks @donmccurdy!

We'd be fine with ether the abstracted blend states or the low level blend functions.

[MiiBond]

From the perspective of Adobe Dimension, we will most likely only need the simple additive and multiplicative blending.

The Adobe Animate team are interested in this extension as well and have more complex uses.
https://helpx.adobe.com/animate/using/applying-blend-modes.html#blend_mode_examples

So, I think Adobe would prefer to keep this full specification rather than simplify it.

[donmccurdy]

I think #822 (comment) makes an argument that alphaMode=MASK should also be allowed with this extension, not just BLEND. @erich666 or @AurL do you have use cases for alpha cutout/mask alongside blending?

[donmccurdy]

Combining alpha blending with alpha cutoff seems practical, and probably worth including with this extension... If so: core material alphaMode may be BLEND or MASK. If set to MASK, both alphaCutoff and blending specified by KHR_blend should be applied.

[pjcozzi]

@donmccurdy is this ready to merge once it has a JSON schema?

@MiiBond have you implemented this? If not, does it look like it will meet your needs and should we wait for you to implement it?

@ggetz did we implement this in Cesium?

[donmccurdy]

There's an implementation of this extension in glTF-Pipeline, at least.

I'm still not really confident whether the low-level blend modes here are the right choice, or whether we want discrete named states like:

Add, Subtract, Multiply, Screen, Divide, Difference, Darken, Lighten, Overlay, Dodge, Burn, Hue, Saturation, Value, Color, Soft Light, Linear Light, ...

Before we merge the extension, additional feedback (@MiiBond, @AurL, others?) would still be welcome. It also seems important to know whether model sources like Sketchfab or authoring tools are likely to implement it, or whether it's just engines interested so far. I think Google Poly will likely use the extension, but need to re-confirm on that.

[donmccurdy]

One other thought – I'm very tempted to pull the GL integer constants out in favor of string enums. In general I think new extensions should likely shift that way.

[lexaknyazev]

I'm very tempted to pull the GL integer constants out in favor of string enums.
I'm still not really confident whether the low-level blend modes here are the right choice

I have the same concerns. Let's discuss on the next call.

---

For the reference, here's the current state of API-level options.

Factors

ZERO
ONE
SRC_COLOR
ONE_MINUS_SRC_COLOR
DST_COLOR
ONE_MINUS_DST_COLOR
SRC_ALPHA
ONE_MINUS_SRC_ALPHA
DST_ALPHA
ONE_MINUS_DST_ALPHA
CONSTANT_COLOR
ONE_MINUS_CONSTANT_COLOR
CONSTANT_ALPHA
ONE_MINUS_CONSTANT_ALPHA
SRC_ALPHA_SATURATE

There are additional four factors for "Dual-Source Blending" that are supported by modern pipelines. They require specific fragment shader outputs, so we cannot expose them.

Operations

FUNC_ADD
FUNC_SUBTRACT
FUNC_REVERSE_SUBTRACT
MIN
MAX

MIN and MAX require OpenGL ES 3.0+ or EXT_blend_minmax.

Advanced Operations

There are 46 more blending operations available in GL_NV_blend_equation_advanced or VK_EXT_blend_operation_advanced extensions. These blending equations do not use source and destination factors. Instead, each blend operation specifies a complete equation based on the source and destination colors.

It's worth noting that Khronos-ratified version of the OpenGL extension (GL_KHR_blend_equation_advanced) exposes only 15 of them. The same 15 operations are available in HTML5 Canvas2D API and will probably be available in WebGL via proposed WEBGL_blend_equation_advanced_coherent extension.

MULTIPLY
SCREEN
OVERLAY
DARKEN
LIGHTEN
COLORDODGE
COLORBURN
HARDLIGHT
SOFTLIGHT
DIFFERENCE
EXCLUSION
HSL_HUE
HSL_SATURATION
HSL_COLOR
HSL_LUMINOSITY

[ggetz]

This is implemented in Cesium as of #1302 (comment) in CesiumGS/cesium#6805.

#1302 (comment) would just need to be addressed.

[MiiBond]

We have not implemented this in Dimension and are unlikely to at this point, I think. My feeling is that it's probably still a useful extension but I'd want the ability to control render order in glTF before it would become really useful.
I prefer the low-level blend modes so that authoring packages have the freedom to choose. It might make it more difficult to author a glTF by hand but is that really a big concern?

[donmccurdy]

Thanks @lexaknyazev, good points. Let's discuss more this week. In that list of 15 blend modes, I'm surprised not to see additive blending. Have I missed something, or are those 15 meant to be an addition to other blend modes?

Thanks for the update @MiiBond! Agreed about render order, and perhaps also depth test for that matter. I wonder if that should be in scope for this extension, or considered in parallel.

I'm not opposed to the low-levelness so much as the possibility that many tools may only support certain discrete blend modes. If users write an arbitrary combination of factors and operations, the result won't load in most DCC tools but might be OK in most engines. And even if we stick with low-level modes, we shouldn't be too OpenGL-specific.

[lexaknyazev]

are those 15 meant to be an addition to other blend modes?

Yes, since they are defined in an extension.

Original NVIDIA-proposed 46 modes include a full set of "high-level" blending operations including those that can be represented with low-level factors/operations.

And even if we stick with low-level modes, we shouldn't be too OpenGL-specific.

I think there are two distinct questions here:

• low-level (factors + operations) vs. high-level (like in Photoshop);
• string enums vs. OpenGL enums vs. our own integer enums.

Just for reference, all OpenGL ES 3.0 blending modes have 1:1 mappings in other graphical APIs.

[donmccurdy]

Closing this PR for the time being. Currently we are more focused on the extensions listed here: https://github.com/KhronosGroup/glTF/tree/master/extensions#in-progress-khronos-and-multi-vendor-extensions

[echadwick-artist]

Posting comments from a recent conversation in the KhronosDevs Slack.

Would be nice to mimic the SparkARStudio BlendMode settings:
https://sparkar.facebook.com/ar-studio/learn/reference/enums/MaterialsModule.BlendMode

I created this chart a few years back when I was developing for the Wii (not using glTF). I found it useful when making shaders in Unity as well. I'd love to see this available in glTF someday, so powerful.

[mikeskydev]

Is there any interest in reviving an extension like this? KHR_techniques_webgl is now retired, and nearly 5 years later there's still no officially supported way for different types of blending, as far as I can see.

[andybak]

Hi. I'd like to add my voice in favour of needing additive blending. I think reducing the scope of this to being purely about additive would maybe help move things forward. Other modes have far less utility and are used much less commonly.

My strongest argument would be that it is present as a core option in nearly every material editor I've ever used.

It's essential to recreate many common effects and far beyond what has been referred to as "particle effects" (itself an incredibly broad area in terms of application)

Along with unlit - additive is used for many things where you want to model elements that aren't solid surfaces. It can be used for light beams, fog, a cheap glow/bloom, planet atmospheres, halos, fire and so on.

It's critical for us on Open Brush - most existing artworks make extensive use of additive brushes. But I suspect we're not alone in this respect.