Max bone influences per vertex

[donmccurdy]

This is a tracking issue for supported bone influences per vertex in the glTF ecosystem. We can move it somewhere more appropriate as needed.

Gist: https://gist.github.com/donmccurdy/4cad2039360fbd7cd55d18b3f0428581

[donmccurdy]

From @lexaknyazev's suggestion in KhronosGroup/glTF-Blender-Exporter#167

[shakesoda]

Unity supports 4 bones per vertex, to the best of my knowledge.

(dunno if you're interested in obscure implementations like mine, but mine/love3d also supports 4)

edit: I'm actually not aware of any implementation of gpu skinning that supports >4, but all cpu skinning implementations seem to.

[najadojo]

Windows Mixed Reality supports only 4 bones per vertex.

[shakesoda]

per the docs, looks like Godot supports 4 bones per vertex.

[lilleyse]

Cesium is 4.

[donmccurdy]

Thanks! This is starting to look like a very consistent list. 🤔

[meshula]

Film and research pipelines use four to eight commonly. Film and research people want to view their assets via gltf too :) At the moment I'm expecting to use a second weight set for excess weights as discussed in other threads, but TBH it feels like a hack, and without support being front and center it'll continue to feel hacky I'm sure. In the UsdSkel spec, we decided not to impose a limit, but instead to impose that the weights have a uniform valence across a vertex group, and that they be ordered so that they can be trivially reduced to a lower valence if desired. https://graphics.pixar.com/usd/docs/api/_usd_skel__schemas.html#UsdSkel_BindingAPI_JointInfluences

[lexaknyazev]

@meshula
glTF spec doesn't formally limit the number of influences, e.g., one can use JOINTS_0, JOINTS_1, JOINTS_2 to provide 12 joints per vertex. We just say that engines don't have to support more than 4.

they be ordered

Do you mean that they are sorted by weight? I think we can impose such restriction on assets with 8 (and more) influences.

Also related: #1213.

[takahirox]

(Regardless whether we impose the 'ordered' restriction or not) I suggest adding two implementation notes

1. up to 4 influences is recommended to let the assets be loaded by most of engines without any problems

2. how engines should handle many influences. For example, picking up largest 4 (max num for the engine) and normalize Support splitting primitives according to max number of uniforms glTF-Blender-Exporter#167 (comment)

[lexaknyazev]

With proposed restriction

picking up largest 4 and normalize

will be

use JOINTS_0 and normalize

Otherwise, an engine loading an asset using 8 influences would need to manually pick largest 4 for each vertex which isn't runtime-friendly.

[takahirox]

Yes, the note and loader's impl will be easier if we impose the restriction.

[meshula]

Other associations to multiple sets are explicit, for example if there are two uv sets, a normal map texture will explicitly reference the appropriate texture coordinate set. With the JOINTS_* convention, it seems there might likewise be an explicit binding, to reference say JOINTS_0 and JOINTS_1. It is a solution to clearly state in the implementation notes that JOINTS_0 and JOINTS_1 are to be concatenated. Perhaps a more rigorous solution would be amend the mesh attributes table with that information? #901

[donmccurdy]

We've discussed implementing support for 8 influences in three.js, and (at the time) assumed we would do so with an additional vec4 attribute. So the JOINTS_0 ... JOINTS_N arrangement is deliberate, and won't necessarily be concatenated.

[lexaknyazev]

won't necessarily be concatenated.

What do you mean by that? An engine supporting 8 influences would probably use two more attributes so that JOINTS_0 and JOINTS_1 data will end up in the same vertex shader (or in subsequent passes).

[donmccurdy]

I thought perhaps @meshula had assumed that using JOINTS_1 "feels like a hack" because in film and research applications there might be no need to split them into two vec4 attributes, and I was just clarifying that this split actually does have a runtime purpose. If that was in fact the purpose. :) But yes they would presumably end up in the same vertex shader, or literally be concatenated for viewers that do skinning on the CPU.

[meshula]

@donmccurdy From the perspective of splitting the weights into vec4 attributes, and the mission of gltf of being well matched to GL, it makes perfect sense, if that was in fact the purpose :) TBH, I had not considered it from that perspective since my skinning is done on CPU as you guess.

To clarify "feels like a hack", since it sounds somewhat inflammatory :) The semantic usage of JOINTS_1 seems to be implicit, whereas the semantic usage of say TEXCOORD_1 will be explicit because the mesh attributes table will tell me that it is to be used for normal mapping or somesuch. So it feels like a hack because the knowledge of JOINTS_1 interpretation is retrieved from forum threads as opposed to the parsed file itself, or here - https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/README.md#skins

Anyway, I'm glad this thread exists to consolidate all the conversation around weights into one place!

[donmccurdy]

glTF spec doesn't formally limit the number of influences, e.g., one can use JOINTS_0, JOINTS_1, JOINTS_2 to provide 12 joints per vertex. We just say that engines don't have to support more than 4.

An engine supporting 8 influences would probably use two more attributes so that JOINTS_0 and JOINTS_1 data will end up in the same vertex shader (or in subsequent passes).

We've said this in threads before, but I just noticed the spec seems to say the opposite:

The number of joints that influence one vertex is limited to 4, so referenced accessors must have VEC4 type ...

[lexaknyazev]

We've said this in threads before, but I just noticed the spec seems to say the opposite:

I'd say that this was overlooked (probably by me) when we have changed semantic name JOINT to JOINTS_0. In the other place, the spec says

Client implementations must support at least ... one joints/weights set.

[donmccurdy]

Moved the data to a Gist (https://gist.github.com/donmccurdy/4cad2039360fbd7cd55d18b3f0428581), I don't think we need to keep this issue open just to record engines' limits.

[dmarnel]

I'm having trouble importing a skinned rig into Oculus Home. It looks like vertices with more than 1 or 2 bone influences cause the skin to deform badly.

[donmccurdy]

@dmarnel if the model works correctly in other glTF viewers (e.g. https://gltf-viewer.donmccurdy.com/, https://sandbox.babylonjs.com/) I think you'll probably need to report an issue to Oculus.

[ninjz]

Is it correct to say single a set of weights per node corresponds to 4 weight influences per node? I'm using Sceneview/sceneform-android which uses Filament under the hood. I've seen here that: "Filament supports only a single set of weights per node (this conforms to the glTF minspec)" so I'm wondering does that mean Filament supports 4 weight influences or 1.

I exported my model through blender exporting all weight influences and it runs fine on BabylonJS sandbox but fails to load/crashes inside of Filament.

[donmccurdy]

@ninjz I think that's correct. I'd probably phrase that as "weights per primitive" rather than "weights per node".

• a node may reference 1 skin and 1 mesh
• a mesh may contain N primitives
• a primitive is a collection of triangles (or points or lines) storing weight influences in vertex attributes
• each set of 4 joints/weights comes at a cost of 2 vertex attributes

Realtime clients are usually constrained in how many vertex attributes they can allocate (often ≤16 in total, shared by other features the primitive needs) but are required to support at least 4 weights (2 vertex attributes, or 1 "set" of weights here) for skinning. Other implementations of skinning may not depend on vertex attributes but this is the most common situation.

[fire]

Godot 4.0 (beta) supports 8 bone influences.

[Thebluedaredevil]

Unity supports up to 32 bone influences per vertex. This is a fact. glTF is AWESOME. This is also fact.

[fire]

@Thebluedaredevil unity supports unlimited according to https://gist.github.com/donmccurdy/4cad2039360fbd7cd55d18b3f0428581