Rendering Modularization: Shader Refactor

[mwilsnd]

Bringing this discussion on to GH, sharing my current findings, documentation and design intentions for shaders.

As part of implementing the proposed shader changes, one of our objectives is to simplify the way shaders work and make it possible for users outside the toolkit to create and replace the built-in programs. Existing programs are heavily templated and the associated layer rendering implementations rely on some of these template parameters.

Take a look at how things are done currently:

• A top-level program inherits mbgl::Program, providing uniform, attribute and texture layouts via template parameters
• mbgl::Program instantiates the actual shader object (gfx::Program) by running context.createProgram<TopLevelClass>
  • gfx::Context::createProgram<T> calls the free-function Backend::Create<Type::OpenGL, ...>, backend template-parameter is hard-coded
  • In a totally different TLU, selected by template specialization and return type overloading, a unique_ptr<gl::Program<T>> is finally created
• gl::Program manages shader creation & permutation, implements the actual draw call, configures the fixed-function state and locates the shader source-code (programs::gl::ShaderSource<T>::vertexOffset, also done via template specialization and hard-coded to read from the blob)
• gl::Program and gfx::Program both accept, as a template parameter, the top-level class inheriting mbgl::Program so they can see the program-specific template parameters (consuming parameters, defined here).

With the addition of a generic base for mbgl::Program<> (gfx::Shader), we can store all programs in a registry as gfx::Shader instances. Our hierarchy would like this:

We can then include a shader registry side-by-side with the existing program collection mbgl::Programs in RendererStaticData like so:

An additional method, mbgl::Programs::registerWith(ShaderRegistry&) will serve to 'install' the built-in shaders in the registry instance. This would look like so:

void Programs::registerWith(gfx::ShaderRegistry& registry) {
    registry.registerShader(
        std::make_shared<BackgroundProgram>(context, programParameters));

If we want to pass shader source ourselves instead of loading from the blob, we can pass it via ProgramParameters, perhaps like so:

registry.registerShader(
        std::make_shared<FillProgram>(context, programParameters
            .withShaderSource("VertexGLSL", "FragmentGLSL")));

In this way, we can fallback to loading from the blob if the optional source strings in programParameters are empty.

So far, so good. But now, what if we want to add a new uniform? Maybe a new texture? These are done via template parameters and would thus require implementing a new type derived from mbgl::Program. That also means creating a new template specialization for gl::Program<Name> to create the new type. Layer rendering also refers to programs (ie: FillProgram) by type, meaning to change the type of program for fill layers we have to change the render_fill_layer implementation. Currently, I feel the only approach to supporting this is waiting for drawables to replace the existing render_x_layer approach. Users replacing programs outright will most likely require this anyways to bind additional textures, set new uniforms or whatever it is they've added to the shader.

I'm curious what others think about this approach. Should we try and flatten the program hierarchy or keep it mostly unchanged? Is there value in being able to change program source without (currently) replacing the program class & implementation (uniforms, textures, paint props)? And should we try and change the gfx::Program-mbgl::Program composition-by-template-specialization mechanism?

[sjg-wdw]

I can explain how we do this in WhirlyGlobe-Maply if that helps.

The short version is we broke out all the uniform types we care about and require that they be wired up for a draw call. This is a very OpenGL ES 2.0 approach. Using Uniform Buffers would be superior and we do something like that with Metal.

Good things:

• Very flexible.
• You can add new uniforms without the library knowing.
• Also vertex arrays, but that tends to be more complex.
• It's easy to mix different levels of uniforms from different parts of the system.

Bad things:

• String comparison gets slow, so we moved to indexed strings.
• Can have programs with dangling uniforms. Typically nothing shows up.
• Have to implement all of that uniform setting logic.

With Metal and GLES we found that there's not much to be shared. We could have built a named uniform mechanism for Metal, but it isn't how Metal wants to work and it would slow things down. Point being your top level Program or Shader object is either going to be very bland or force a lot of work on your subclasses.

As an example of how this plays out, let's say we have a custom shader for Metal and for OpenGL ES. Maybe a particle shader, as it's a simple idea. You have to feed in speed and size and texture and all that other good stuff.

Texture setting would have a generic top level method in the Program with implementations in each subclass for a rendering SDK. But uniform setting would not necessarily. On iOS we'd pass in a struct wrapped in an NSData. On Android we'd call out individual uniforms by name.

If we decide that functionality is cross platform, then we build a high level interface on each SDK implementation and expose a generic set of options to be manipulated. This also lets you hide threading details and such. Because if you try to manipulate uniforms on the main loop on Android, it won't go well. There are different reasons that won't work with Metal, like frames in flight.

I'm not sure I'm suggesting the named uniform interface here. Perhaps we could require UBO support for anything new, which syncs up well with Metal and make that interface as flexible as it needs to be. If you're passing buffers around, it doesn't need to be very.

Then if you don't have UBO support you're back on the old shaders and the old layers.

[mwilsnd]

Moving to ES 3, we should be safe to require UBOs exclusively. If we go that route, there isn't actually that much that would change in the template layer. Instead of passing an indexed tuple of uniforms, we pass a struct type that can be copied to GPU memory. That means deleting/contending with UniformStates, changing the draw method to update the UBO and changing the logic of the various layoutUniformValues methods on the various programs. This same pattern of passing a TypeList of uniforms is repeated for named texture samplers and named vertex attributes, with PaintProperties selecting which uniforms are to be additionally data-driven (possibly requiring a new shader permutation).

The top-most level shader class currently exists for a few reasons,

• To use for the mentioned template specializations (sharing template parameters with gl::Program, deducing shader source location)
• To describe the vertex and uniform layouts
• To further specialize the generic programs

All the graphics API specific stuff is hidden in that gfx::Program derived member of the top-level program.