Add a Material allowing scripted generation of shader code

[Damnae]

Problem

Simply creating shaders for each type of visual effects rapidly becomes a mess of all the options you could want depending on what effect you're trying to achieve:
Sometimes I'll need 1, 2 or 3 textures, sometimes it's additive, sometimes it's mix blending, sometimes I want soft "particles" and sometimes i want billboard (in various flavors), sometimes the 2nd texture's color is multiplied with the first and sometimes it's used as alpha. sometimes the 3rd texture distorts UV and sometimes I want it to be used as a palette, sometimes the textures scroll by time, sometimes I need them to be by instance so I can control the scrolling...

Depending on the effect I'll need various combinations of all of the above and more. It quickly turns into a big mess and the shader names become a light novel trying to describe what that particular shader's fixed options are... I've tried using .gdshaderinc but while it reduces the copypasting it doesn't really fix the problem.

Solution?

What is needed is something like a StandardMaterial3D, that generates a shader depending on the selected options.
While StandardMaterial3D has a lot of options for "normal" materials, when making materials for visual effects, it needs a different set of options (see Problem section for examples).

Because the options a "VisualEffectMaterial3D" should contain would be different for every project, we should be able to extend Material into our own "VisualEffectMaterial3D", like StandardMaterial3D does.

---

I initially thought I could just extend ShaderMaterial, and while the early results were promising (https://streamable.com/pls2va), there are some issues:

• There's no way to tell when the shader is needed, so it has to be regenerated every time a property changes. On load since the properties are changed 1 by 1 by godot, it has to regenerate it a lot of times. Internally godot seem to be handling this through overriding _get_rid/_get_shader_rid/etc. and having a flush method called by the engine. It does not appear possible from gdscript/c#. Ideally you'd have a method to mark the shader as "dirty" that you'd call when one of the properties is modified, and you could override a method that would be called by the material when it needs the new shader code (just like the current material do). This is the main thing that I don't see possible without a Material class made for this purpose.
• If you open the shader in the shader editor, the material will stop responding to changes because the editor is overriding it every frame, the only fix is to close the editor, remove the shader and change some option to regenerate it. From there it will create an error message every time the material is saved, for every time the shader was cleared this way. A Material class made for this purpose could simply not expose its shader, and allow itself to be converted to a text shader like StandardMaterial3D/etc.
• Similar shaders (generated by the same set of properties but able to use different shader properties) are not shared between materials like StandardMaterial3D/etc. does. A Material class made for this purpose may be able to help with this.
• C# Tool doesn't behave properly when the solution is rebuilt, and I had to rewrite everything in gdscript. This is not really the point of this discussion though.

VfxMaterial.zip Here's the material I made for this if you'd like to give it a try.

[Calinou]

To my knowledge, this is already possible but not trivial. cc @QbieShay who I remember implementing something like this in a project.

[Damnae]

After spending more time on this, I believe my issues with making this work properly are centered around C# [Tool] breaking when the solution is rebuilt, resulting on settings being lost, shader not being updated or code looking cursed in order for its data to survive the Build reload.

[QbieShay]

Ah :/ in that case may be worth opening an issue for C#?

[Damnae]

I wrote something about the c# issues, maybe not at the best place #4007 (comment); tbh I don't know how these could be solved, unless some kind of c# hot reload is implemented. The current way of overwriting everything that's Variant compatible (even private members) does not work well for c# [Tool]s.

But I'd like to focus on this, so I'm slowly rewriting my material in gdscript.

[QbieShay]

Awesome! Best of luck with that ✨ and im always happy to see cool shader stuff so feel free to join us and post your progress on discord! https://discord.gg/HX4xAGaGjm

[Damnae]

Now that I've rewritten the material in gdscript, I've edited the main post with the remaining issues I see in not having a class made for this.