WebGL linear backbuffer and wide gamut output

[kainino0x]

Currently, all output of WebGL rendering is either treated as if it's in sRGB or in the display's color space (depending on browser and OS). That is, no conversion is done between fragment shader output and the values sent to the screen.

For linear rendering pipelines - in which rendering is done in linear space - it would be beneficial for engines/applications to be able to output their rendering results from the fragment shader in linear space, and have this remapped this into the display's color space. Currently, the application must do its own conversion from linear space to sRGB. (Plus this doesn't allow applications to take advantage of wide-gamut displays.)

For Canvas, a color space proposal exists here:
https://github.com/WICG/canvas-color-space/blob/master/CanvasColorSpaceProposal.md
I expect that color space support will be added to WebGL canvases in a very similar way.

[dmikis]

Ability to specify pixel format of the default framebuffer with wider channels than RGBA8 may be beneficial too, I think.

[juj]

This keeps coming up every few months, would be great to have native support for this.

[MarkCallow]

Actually WebGL has a "linear" backbuffer which is why no conversion is done to the fragment shader outputs. I put linear in inverted commas because no conversion is done between the backbuffer and the display so whatever values are written are interpreted as in the display's color space as @juj noted. A true linear backbuffer would have an sRGB or display encoding step between the buffer and the display.

What are not currently supported by WebGL and should be are sRGB framebuffers where the hardware does sRGB encoding of the fragment shader outputs. This would be seemingly trivial to support, given virtually all hardware does it today, and would avoid applications having to do encoding in their fragment shaders. It would also provide correct blending results, something impossible to achieve today. Backbuffer content must be converted to linear before blending then converted back again.

If WebGL did provide a true linear 8-bit backbuffer there is a good chance the people making the requests @juj is hearing would be unhappy because they would start to see banding in their results especially if using blending. I think what they really want are sRGB framebuffers.

[lexaknyazev]

sRGB framebuffers where the hardware does sRGB encoding of the fragment shader outputs

They technically exist via an extension for WebGL 1.0 and as a part of WebGL 2.0 core, but only as FBO attachments. There's no way to explicitly set the transfer function of the backbuffer, though.

Still, it's probably possible to have hardware-based sRGB conversion for output in WebGL 2.0:

1. Create an FBO attachment with sRGB pixel format.
2. Output linear values from fragment shaders to it; sRGB encoding will happen automatically, blending will also work correctly.
3. Read sRGB data into a PIXEL_PACK_BUFFER, rebind the pixel buffer as PIXEL_UNPACK_BUFFER.
4. Fill a 2D texture from PIXEL_UNPACK_BUFFER; render the texture into the backbuffer as a full-screen quad without filtering.

[kenrussell]

@lexaknyazev that's clever but probably expensive in terms of memory bandwidth.

Discussions are actively ongoing about how to specify the color space and bit depth of canvases' backing stores. The current proposal is here:

https://github.com/WICG/canvas-color-space

more specifically:

https://github.com/WICG/canvas-color-space/blob/master/CanvasColorSpaceProposal.md

Please review, comment and file any issues you see. Thanks!

[lexaknyazev]

expensive in terms of memory bandwidth

Yeah, I'd prefer something like blit operation with sRGB conversion disabled or low-level buffer-style renderbuffer copy.

Anyway, there are two issues here:

• lack of HTML5 Canvas color management (partly covered by the WICG proposal);
• lack of ability to "override" the transfer function of existing GL objects to reduce the number of buffer copies (like in EXT_sRGB_write_control and EXT_texture_format_sRGB_override).

For now, the most optimal way of color-correct rendering with blending seems to be:

1. Create an FBO texture attachment with sRGB or float pixel format (another good alternative would be not yet exposed unorm16).
2. Output linear values from fragment shaders to it; do as many reads/writes/blending as needed, sRGB transfers (if any) will happen automatically.
3. Render the texture into the backbuffer as a full-screen quad with final linear-to-sRGB transfer in the fragment shader.

@kenrussell that WICG repo and its linked discourse forum look stalled. Where should issues be filed?

[kenrussell]

Please file issues or put up pull requests again on https://github.com/WICG/canvas-color-space . I'll try to help ensure the appropriate people look at them and respond.

[Tronic]

Since the output canvas gets further composited by browser, it should be kept in linear alpha-premultiplied form, and converted to screen colour space by the browser only once it is finished with all the composition.

Central issues:

• All blending (incl. CSS opacity and such) should be done in linear space, and will always have premultiplied output
  • Correct blending with sRGB buffers requires repeated conversions back and forth
  • OpenGL sRGB framebuffers do such conversion on every blending operation
• Linear to sRGB conversion only works correctly with unpremultiplied alpha, or without transparency
  • Dividing by alpha to get there is problematic
• Inputs need to be converted
  • sRGB textures
  • CSS hex colors
  • GL floating-point colours? (otherwise 0.25, 0.25, 0.25 input is taken as linear and becomes #888 sRGB)

Most existing software use "linear" processing, completely ignoring these issues although they actually work in non-linear in display device's colour space. Doing it properly will necessarily affect existing software, and could only be partially hidden behind a configuration option.