Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Add reusable shader functions for transforming position/normal/tangent #4901

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions assets/shaders/animate_shader.wgsl
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
[[group(1), binding(0)]]
var<uniform> mesh: Mesh;

// NOTE: Bindings must come before functions that use them!
#import bevy_pbr::mesh_functions

struct Vertex {
[[location(0)]] position: vec3<f32>;
[[location(1)]] normal: vec3<f32>;
Expand All @@ -17,10 +20,8 @@ struct VertexOutput {

[[stage(vertex)]]
fn vertex(vertex: Vertex) -> VertexOutput {
let world_position = mesh.model * vec4<f32>(vertex.position, 1.0);

var out: VertexOutput;
out.clip_position = view.view_proj * world_position;
out.clip_position = mesh_position_local_to_clip(mesh.model, vec4<f32>(vertex.position, 1.0));
out.uv = vertex.uv;
return out;
}
Expand Down
17 changes: 9 additions & 8 deletions assets/shaders/custom_vertex_attribute.wgsl
Original file line number Diff line number Diff line change
@@ -1,28 +1,29 @@
#import bevy_pbr::mesh_view_bindings
#import bevy_pbr::mesh_bindings

struct Vertex {
[[location(0)]] position: vec3<f32>;
[[location(1)]] blend_color: vec4<f32>;
};

struct CustomMaterial {
color: vec4<f32>;
};
[[group(1), binding(0)]]
var<uniform> material: CustomMaterial;

// NOTE: Bindings must come before functions that use them!
#import bevy_pbr::mesh_functions
Comment on lines +10 to +11
Copy link
Contributor

@Weibye Weibye Jun 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit / curious: What is the reasoning for this import statement to not be at the top of the file like the rest?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my mind, the comment is directly about the position of this one import in relation to the others and the types and bindings in the file. As such, it made sense to me to add the note on it.

Copy link
Contributor

@Weibye Weibye Jun 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the comment I get. To clarify, why this:

#import X
[bindgroup(A)]

// Some comment
#import Y
[bindgroup(B)]

Instead of this:

#import X;
// Some comment
#import Y;

[bindgroup(A)]

[bindgroup(B)]

(coming from other languages, used to seeing all import statements at the top of the file, so I'm trying to understand the norms and patterns at play)

Copy link
Contributor Author

@superdump superdump Jun 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s what the comment is supposed to explain. Types and bindings must come before functions that use them. The import statement that the comment is about is importing ‘bevy_pbr::mesh_functions’ which contains a bunch of helper functions and those functions refer to members of the view and mesh bindings.

If you understand ^ that explanation, do you have a suggestion for what could make the comment clearer?

Copy link
Contributor

@Weibye Weibye Jun 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your comment is both clear and understandable :) Placing the comment + import statement and the usage of the function makes completely sense in this case.

This is more me trying to understand wgsl (new to me). I was curious that perhaps there was something in wgsl that required the import statement to placed close to where it was going to be used, but I understand now that as long as it is placed before, it could be placed anywhere.


struct Vertex {
[[location(0)]] position: vec3<f32>;
[[location(1)]] blend_color: vec4<f32>;
};

struct VertexOutput {
[[builtin(position)]] clip_position: vec4<f32>;
[[location(0)]] blend_color: vec4<f32>;
};

[[stage(vertex)]]
fn vertex(vertex: Vertex) -> VertexOutput {
let world_position = mesh.model * vec4<f32>(vertex.position, 1.0);

var out: VertexOutput;
out.clip_position = view.view_proj * world_position;
out.clip_position = mesh_position_local_to_clip(mesh.model, vec4<f32>(vertex.position, 1.0));
out.blend_color = vertex.blend_color;
return out;
}
Expand Down
7 changes: 4 additions & 3 deletions assets/shaders/instancing.wgsl
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
[[group(1), binding(0)]]
var<uniform> mesh: Mesh;

// NOTE: Bindings must come before functions that use them!
#import bevy_pbr::mesh_functions

struct Vertex {
[[location(0)]] position: vec3<f32>;
[[location(1)]] normal: vec3<f32>;
Expand All @@ -21,10 +24,8 @@ struct VertexOutput {
[[stage(vertex)]]
fn vertex(vertex: Vertex) -> VertexOutput {
let position = vertex.position * vertex.i_pos_scale.w + vertex.i_pos_scale.xyz;
let world_position = mesh.model * vec4<f32>(position, 1.0);

var out: VertexOutput;
out.clip_position = view.view_proj * world_position;
out.clip_position = mesh_position_local_to_clip(mesh.model, vec4<f32>(position, 1.0));
out.color = vertex.i_color;
return out;
}
Expand Down
7 changes: 4 additions & 3 deletions assets/shaders/shader_defs.wgsl
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
[[group(1), binding(0)]]
var<uniform> mesh: Mesh;

// NOTE: Bindings must come before functions that use them!
#import bevy_pbr::mesh_functions

struct Vertex {
[[location(0)]] position: vec3<f32>;
[[location(1)]] normal: vec3<f32>;
Expand All @@ -16,10 +19,8 @@ struct VertexOutput {

[[stage(vertex)]]
fn vertex(vertex: Vertex) -> VertexOutput {
let world_position = mesh.model * vec4<f32>(vertex.position, 1.0);

var out: VertexOutput;
out.clip_position = view.view_proj * world_position;
out.clip_position = mesh_position_local_to_clip(mesh.model, vec4<f32>(vertex.position, 1.0));
return out;
}

Expand Down
5 changes: 4 additions & 1 deletion crates/bevy_pbr/src/render/depth.wgsl
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ var<uniform> joint_matrices: SkinnedMesh;
#import bevy_pbr::skinning
#endif

// NOTE: Bindings must come before functions that use them!
#import bevy_pbr::mesh_functions

struct Vertex {
[[location(0)]] position: vec3<f32>;
#ifdef SKINNED
Expand All @@ -34,6 +37,6 @@ fn vertex(vertex: Vertex) -> VertexOutput {
#endif

var out: VertexOutput;
out.clip_position = view.view_proj * model * vec4<f32>(vertex.position, 1.0);
out.clip_position = mesh_position_local_to_clip(model, vec4<f32>(vertex.position, 1.0));
return out;
}
8 changes: 8 additions & 0 deletions crates/bevy_pbr/src/render/mesh.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ pub const MESH_TYPES_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2506024101911992377);
pub const MESH_BINDINGS_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 16831548636314682308);
pub const MESH_FUNCTIONS_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 6300874327833745635);
pub const MESH_SHADER_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 3252377289100772450);
pub const SKINNING_HANDLE: HandleUntyped =
Expand All @@ -69,6 +71,12 @@ impl Plugin for MeshRenderPlugin {
"mesh_bindings.wgsl",
Shader::from_wgsl
);
load_internal_asset!(
app,
MESH_FUNCTIONS_HANDLE,
"mesh_functions.wgsl",
Shader::from_wgsl
);
load_internal_asset!(app, MESH_SHADER_HANDLE, "mesh.wgsl", Shader::from_wgsl);
load_internal_asset!(app, SKINNING_HANDLE, "skinning.wgsl", Shader::from_wgsl);

Expand Down
31 changes: 10 additions & 21 deletions crates/bevy_pbr/src/render/mesh.wgsl
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#import bevy_pbr::mesh_view_bindings
#import bevy_pbr::mesh_bindings

// NOTE: Bindings must come before functions that use them!
#import bevy_pbr::mesh_functions

struct Vertex {
[[location(0)]] position: vec3<f32>;
[[location(1)]] normal: vec3<f32>;
Expand Down Expand Up @@ -35,35 +38,21 @@ fn vertex(vertex: Vertex) -> VertexOutput {
var out: VertexOutput;
#ifdef SKINNED
var model = skin_model(vertex.joint_indices, vertex.joint_weights);
out.world_position = model * vec4<f32>(vertex.position, 1.0);
out.world_normal = skin_normals(model, vertex.normal);
#ifdef VERTEX_TANGENTS
out.world_tangent = skin_tangents(model, vertex.tangent);
#endif
#else
out.world_position = mesh.model * vec4<f32>(vertex.position, 1.0);
out.world_normal = mat3x3<f32>(
mesh.inverse_transpose_model[0].xyz,
mesh.inverse_transpose_model[1].xyz,
mesh.inverse_transpose_model[2].xyz
) * vertex.normal;
#ifdef VERTEX_TANGENTS
out.world_tangent = vec4<f32>(
mat3x3<f32>(
mesh.model[0].xyz,
mesh.model[1].xyz,
mesh.model[2].xyz
) * vertex.tangent.xyz,
vertex.tangent.w
);
var model = mesh.model;
#endif
out.world_position = mesh_position_local_to_world(model, vec4<f32>(vertex.position, 1.0));
out.world_normal = mesh_normal_local_to_world(vertex.normal);
out.uv = vertex.uv;
#ifdef VERTEX_TANGENTS
out.world_tangent = mesh_tangent_local_to_world(model, vertex.tangent);
#endif
#ifdef VERTEX_COLORS
out.color = vertex.color;
#endif

out.uv = vertex.uv;
out.clip_position = view.view_proj * out.world_position;
out.clip_position = mesh_position_world_to_clip(out.world_position);
return out;
}

Expand Down
36 changes: 36 additions & 0 deletions crates/bevy_pbr/src/render/mesh_functions.wgsl
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#define_import_path bevy_pbr::mesh_functions

fn mesh_position_local_to_world(model: mat4x4<f32>, vertex_position: vec4<f32>) -> vec4<f32> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure I've endorsed this idea in the past, but having thought about it a bit more, I think I'd like to push back on this general pattern a bit. Especially for the functions that just alias multiplication. Is introducing special-case methods for those operations "worth it"?

On the one hand, they give descriptive (and consistent) names to the operations (mesh_position_local_to_world, mesh_position_world_to_clip), etc.

On the other hand, teaching people to use this:
mesh_position_local_to_world(model, position)
over this:
model * position

Feels wrong to me. It is abstracting out a fundamental concept (vec/matrix multiplication) in favor of a custom case-specific concept. Worse, that case-specific concept / naming doesn't hold if you pass a non-model matrix in (or non mesh position). And on top of that, it requires you to pull in an import that you might not otherwise need.

I think we should only provide helper functions that actually reduce boilerplate and/or fully abstract out concepts:

  • Functions that abstract out multiple fundamental operations, such as mesh_position_local_to_clip replacing a "world position" operation followed by a "world to clip" operation.
  • Functions that fully abstract out bindings. On that topic, I think the UX of mesh_position_local_to_clip would benefit from abstracting out the model matrix and taking a single vec as input (like Unity does in their shaders). That makes it dead-simple to call and better justifies the abstraction cost. I think I see why we aren't doing that here (avoiding double-calculating mesh skinning?). But that seems like a technical hurdle worth solving (maybe not in this pr, but at some point).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note a complete response, just a quick one. The value I see for the local to world case is that it creates consistency and mental clarity that you’re doing it in the right way according to the engine regardless of whether your calculating the world or clip space versions. You could argue that someone should know that projection * inverse view * model * local position = clip position, and I would agree. But when doing a depth prepass (which I am expecting we will at some point) and using the equal depth comparison function in the main pass, float precision of the calculations will matter and I wouldn’t expect someone to know that. And then if you don’t have a function for local to world but do for local to clip, then there’s some inconsistency in the story. There’s nothing stopping people just using the matrices in the bindings, but if code examples use these functions then people will be encouraged to use them. When I move the mesh uniform into an instance buffer for performance reasons, you anyway won’t be able to just refer to mesh.model and will need to reconstruct the model matrix.

Maybe we can use some ‘global variables’ to cache calculation of matrices somehow so that you only pass in the vector to be transformed…? I don’t know if wgsl would allow that.

Copy link
Member

@cart cart Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we actually abstracted out the mesh matrix (enabling things like abstracting out the instance buffer access in the future) id be more convinced. I still think as it stands the "consistency" of mesh_position_local_to_world doesn't justify its complexity cost. As written, it doesn't actually help us with abstracting out things like "mesh matrix sources". And it makes it too easy to use the function in the "wrong" way (by passing in the wrong matrix).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just verified that this is possible:

var<private> MESH_MODEL: mat4x4<f32>;

fn mesh_position_local_to_world(vertex_position: vec4<f32>) -> vec4<f32> {
    return MESH_MODEL * vertex_position;
}

fn update_mesh_model(vertex: Vertex) {
    #ifdef SKINNED
    skin_model(vertex.joint_indices, vertex.joint_weights);
    #else
    MESH_MODEL = mesh.model;
    #endif
}

It would require calling update_mesh_model(vertex) at the start of the vertex shader. But it would also provide the benefit of fully abstracting out skinning for most use cases.

Another downside is that this requires doing the #import bevy_pbr::mesh_functions call after defining Vertex.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also do lazy calculations with a second var<private> MESH_MODEL_CALCULATED: bool;
But I'm hesitant to add runtime calculations for what should really be static.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also use the godot approach: treat user-defined vertex + fragment functions as "internal":

[[stage(vertex)]]
fn internal_vertex(vertex: Vertex) -> VertexOutput {
  /* do setup work here, such as setting MESH_MODEL */
  return vertex(vertex);
}

// user defines this, which will implicitly get injected in to a template containing `internal_vertex` / whatever else is needed
fn vertex(vertex: Vertex) -> VertexOutput {
  // user is free to assume MESH_MODEL is already valid and
  // call things like mesh_position_local_to_world(vertex.position)
}

Definitely more abstraction (and more "fixed"). Just throwing out options to get the juices flowing.
Also, in the interest of moving forward on "callable pbr", im happy to table this, as I'm encouraging some serious scope creep here, and your changes are non-breaking / optional.

But I still feel relatively strongly about the core principles outlined above in the bullet points of this message: #4901 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It maybe doesn’t need to be global. But I don’t know if you like that. I was going to try a ptr with function scope.

return model * vertex_position;
}

fn mesh_position_world_to_clip(world_position: vec4<f32>) -> vec4<f32> {
return view.view_proj * world_position;
}

// NOTE: The intermediate world_position assignment is important
// for precision purposes when using the 'equals' depth comparison
// function.
fn mesh_position_local_to_clip(model: mat4x4<f32>, vertex_position: vec4<f32>) -> vec4<f32> {
let world_position = mesh_position_local_to_world(model, vertex_position);
return mesh_position_world_to_clip(world_position);
}

fn mesh_normal_local_to_world(vertex_normal: vec3<f32>) -> vec3<f32> {
return mat3x3<f32>(
mesh.inverse_transpose_model[0].xyz,
mesh.inverse_transpose_model[1].xyz,
mesh.inverse_transpose_model[2].xyz
) * vertex_normal;
}

fn mesh_tangent_local_to_world(model: mat4x4<f32>, vertex_tangent: vec4<f32>) -> vec4<f32> {
return vec4<f32>(
mat3x3<f32>(
model[0].xyz,
model[1].xyz,
model[2].xyz
) * vertex_tangent.xyz,
vertex_tangent.w
);
}
20 changes: 3 additions & 17 deletions crates/bevy_pbr/src/render/skinning.wgsl
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
// If using this WGSL snippet as an #import, a dedicated
// If using this WGSL snippet as an #import, a dedicated
// "joint_matricies" uniform of type SkinnedMesh must be added in the
// main shader.

#define_import_path bevy_pbr::skinning

/// HACK: This works around naga not supporting matrix addition in SPIR-V
/// HACK: This works around naga not supporting matrix addition in SPIR-V
// translations. See https://github.com/gfx-rs/naga/issues/1527
fn add_matrix(
a: mat4x4<f32>,
Expand All @@ -30,7 +30,7 @@ fn skin_model(

fn inverse_transpose_3x3(in: mat3x3<f32>) -> mat3x3<f32> {
let x = cross(in.y, in.z);
let y = cross(in.z, in.x);
let y = cross(in.z, in.x);
let z = cross(in.x, in.y);
let det = dot(in.z, z);
return mat3x3<f32>(
Expand All @@ -50,17 +50,3 @@ fn skin_normals(
model[2].xyz
)) * normal;
}

fn skin_tangents(
model: mat4x4<f32>,
tangent: vec4<f32>,
) -> vec4<f32> {
return vec4<f32>(
mat3x3<f32>(
model[0].xyz,
model[1].xyz,
model[2].xyz
) * tangent.xyz,
tangent.w
);
}
25 changes: 13 additions & 12 deletions crates/bevy_pbr/src/render/wireframe.wgsl
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
#import bevy_pbr::mesh_types
#import bevy_pbr::mesh_view_bindings

[[group(1), binding(0)]]
var<uniform> mesh: Mesh;

#ifdef SKINNED
[[group(1), binding(1)]]
var<uniform> joint_matrices: SkinnedMesh;
#import bevy_pbr::skinning
#endif

// NOTE: Bindings must come before functions that use them!
#import bevy_pbr::mesh_functions

struct Vertex {
[[location(0)]] position: vec3<f32>;
#ifdef SKINNED
Expand All @@ -9,19 +21,10 @@ struct Vertex {
#endif
};

[[group(1), binding(0)]]
var<uniform> mesh: Mesh;

struct VertexOutput {
[[builtin(position)]] clip_position: vec4<f32>;
};

#ifdef SKINNED
[[group(1), binding(1)]]
var<uniform> joint_matrices: SkinnedMesh;
#import bevy_pbr::skinning
#endif

[[stage(vertex)]]
fn vertex(vertex: Vertex) -> VertexOutput {
#ifdef SKINNED
Expand All @@ -30,10 +33,8 @@ fn vertex(vertex: Vertex) -> VertexOutput {
let model = mesh.model;
#endif

let world_position = model * vec4<f32>(vertex.position, 1.0);
var out: VertexOutput;
out.clip_position = view.view_proj * world_position;

out.clip_position = mesh_position_local_to_clip(model, vec4<f32>(vertex.position, 1.0));
return out;
}

Expand Down
8 changes: 8 additions & 0 deletions crates/bevy_sprite/src/mesh2d/mesh.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ pub const MESH2D_TYPES_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 8994673400261890424);
pub const MESH2D_BINDINGS_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 8983617858458862856);
pub const MESH2D_FUNCTIONS_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 4976379308250389413);
pub const MESH2D_SHADER_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2971387252468633715);

Expand Down Expand Up @@ -72,6 +74,12 @@ impl Plugin for Mesh2dRenderPlugin {
"mesh2d_bindings.wgsl",
Shader::from_wgsl
);
load_internal_asset!(
app,
MESH2D_FUNCTIONS_HANDLE,
"mesh2d_functions.wgsl",
Shader::from_wgsl
);
load_internal_asset!(app, MESH2D_SHADER_HANDLE, "mesh2d.wgsl", Shader::from_wgsl);

app.add_plugin(UniformComponentPlugin::<Mesh2dUniform>::default());
Expand Down
24 changes: 7 additions & 17 deletions crates/bevy_sprite/src/mesh2d/mesh2d.wgsl
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#import bevy_sprite::mesh2d_view_bindings
#import bevy_sprite::mesh2d_bindings

// NOTE: Bindings must come before functions that use them!
#import bevy_sprite::mesh2d_functions

struct Vertex {
[[location(0)]] position: vec3<f32>;
[[location(1)]] normal: vec3<f32>;
Expand Down Expand Up @@ -28,26 +31,13 @@ struct VertexOutput {

[[stage(vertex)]]
fn vertex(vertex: Vertex) -> VertexOutput {
let world_position = mesh.model * vec4<f32>(vertex.position, 1.0);

var out: VertexOutput;
out.uv = vertex.uv;
out.world_position = world_position;
out.clip_position = view.view_proj * world_position;
out.world_normal = mat3x3<f32>(
mesh.inverse_transpose_model[0].xyz,
mesh.inverse_transpose_model[1].xyz,
mesh.inverse_transpose_model[2].xyz
) * vertex.normal;
out.world_position = mesh2d_position_local_to_world(mesh.model, vec4<f32>(vertex.position, 1.0));
out.clip_position = mesh2d_position_world_to_clip(out.world_position);
out.world_normal = mesh2d_normal_local_to_world(vertex.normal);
#ifdef VERTEX_TANGENTS
out.world_tangent = vec4<f32>(
mat3x3<f32>(
mesh.model[0].xyz,
mesh.model[1].xyz,
mesh.model[2].xyz
) * vertex.tangent.xyz,
vertex.tangent.w
);
out.world_tangent = mesh2d_tangent_local_to_world(vertex.tangent);
#endif
#ifdef VERTEX_COLORS
out.colors = vertex.colors;
Expand Down
Loading