Animation loop behavior

[MarcoHutter]

(Related to #569 to some extent)

Are there any plans to support different looping behaviors for animations? For example, whether animations should be ran once, or in a loop, or in a ping-pong fashion? (This is roughly refering to https://cesiumjs.org/Cesium/Build/Documentation/ModelAnimationLoop.html).

If so, this would raises some questions about the interpretation of the times that are given as the key times for the animation, going beyond the issue linked above. For example, for two animations (sketching the time lines here)

            0    1    2    3    4    5    6
Scene Time  |----|----|----|----|----|----|...
Animation 0           |---------|              
Animation 1      |-------------------|     

it is not clear what the state of Animation 0 should be at the time 1.5 or 4.5. One reasonable assumption is that it should be at its start- or end state (i.e. at the state of 2.0 and 4.0, respectively). For other loop types, it would be equally reasonable to be at the state that it had at 2.5 or 3.5 (or vice versa). Are there any intentions to clarify or specify this in more detail?

---

(Edited to fix a typo: 2.5 instead of 0.5)

[pjcozzi]

Are there any plans to support different looping behaviors for animations?

Not immediately, but this is something that an app could put in an extras property or be proposed as an extension, and then prompted to core spec when it gains enough use.

Are there any intentions to clarify or specify this in more detail?

I agree that the spec should be explicit about out of range times so I labeled this issue 1.0.1 so it can be addressed in the next minor spec update.

My initial thought is to say that all times must be in range; otherwise, the implementation is not valid. This gives the implementation freedom to loop different animations in different ways, albeit two valid renderers could loop differently, but I do not know if this is a pragmatic concern.

[kbirk]

I think a looped parameter indicating how an animation is intended to be looped would be very helpful to consumers of the glTF format and would allow the specification to remove any ambiguity over the intervals of interpolation.

For example, if the looped property is REPEAT, the specification could enforce that channel values for channel.output[0] and channel.output[frames.length-1] MUST be equal.

Otherwise there is ambiguity over the interpolation interval between channel.output[frames.length-1] and channel.output[0]`.

For example, let's say we have a model that has an animation that is clearly intended to be looped with the following TIME inputs and scale outputs:

TIME = [ 0, 0.5, 2.0, 5.0 ]
scale = [ [0, 0, 0], [1, 1, 1], [4, 4, 4], [10, 10, 10] ]

Because scale[0] and scale[3] are not equal, it would indicate that there should be interpolation between frame 3 and frame 0. But the length interval over which to interpolate is completely ambiguous.

Do I average the intervals and interpolate over 1.6 seconds? Do I use the previous interval and interpolate over 3 seconds?

This is occurs in the monster.gltf sample model. The walking animation is clearly intended to be looped, but the first and last channel output values are not equal. The TIME inputs are uniform and the method of averaging the interval is sufficient, but what about cases like the example above where the intervals are not uniform?

[MarcoHutter]

This issue mainly referred to whether it should be possible to define the desired looping behavior at all.

(The proper interpretation of the times is probably well covered with this other issue, where Patrick pointed out the crucial question in #569 (comment) . When it is clear how the "global" time is mapped to the "local" time, it may become clearer how to define looping behaviors - however, if this mapping is not specified yet, then one should keep in mind that different looping behaviors might have to be supported in the future)

For the looping behaviors, there are some caveats. The described semantics of a REPEAT animation are a bit difficult to enforce via the spec. More importantly: What if the user wants the values to jump? If a user wants a looped animation, and he wants the scale to be [10,10,10] after 5.0 seconds, and when wants the values to jump back to [0,0,0] when the loop restarts? I think it should at least be possible to do this.

Further questions, e.g. whether it should be possible to specify the looping behavior for each animation individually (or maybe even for each channel?) will have to be considered as well, but all this will have to be validated carefully when a decision is made of whether the spec will be extended accordingly.

[kbirk]

Good point about an intended jump in animation, that was definitely an oversight on my part.

However enforcing that there is to be no interpolation between frames N-1 to 0 in a looped animation would remove the ambiguity as to whether an animation should jump or not.

For example:

Jumping animation

TIME = [ 0, 2.0 ]
scale = [ [1, 1, 1], [2, 2, 2] ]

Smooth animation

TIME = [ 0, 2.0, 4.0 ]
scale = [ [1, 1, 1], [2, 2, 2], [ 1, 1, 1 ] ]

[pjcozzi]

Actually ran into this general issue in Cesium recently, CesiumGS/cesium#3934.

[lexaknyazev]

Can we allow also non-interpolated samplers (think of frame by frame animation)?
And btw, is TIME the only valid sampler.input value? We can adopt the same naming policy as with semantics (all custom values must start with _).

[pjcozzi]

And btw, is TIME the only valid sampler.input value? We can adopt the same naming policy as with semantics (all custom values must start with

Yes. Can you open a pull request for the spec/schema?

Can we allow also non-interpolated samplers (think of frame by frame animation)?

Yes, if there are sufficient use cases. What are use cases do you have in mind?

Also, labeling this 1.1 so 1.0.1 stays focused on validation, not new features.

[lexaknyazev]

Can we allow also non-interpolated samplers (think of frame by frame animation)?

Yes, if there are sufficient use cases. What are use cases do you have in mind?

Hard key frames on camera position/rotation to make "cuts".

Also such technique could be used to avoid runtime cost of interpolation (if we pre-sample matrices for each frame). Eternal trade off: computing or storage/bandwidth.

[pjcozzi]

Hard key frames on camera position/rotation to make "cuts"

Sounds potentially useful. Are you sure that this is commonly done in one animation, e.g., not two separate animations?

We need to be careful to not let feature-creep make the glTF spec complex; everything needs to be supported by widely needed use cases, and then corner cases can be handled by extensions or application-specific extras.

[lexaknyazev]

Are you sure that this is commonly done in one animation, e.g., not two separate animations?

Of course not. Real-world cinematic camera movements/cuts are very complex and can consist of several consequential non-linear animations.

At the current state of spec, sampler.interpolation is always LINEAR with no chance of turning it off. I propose to allow NONE (or DISABLED) as a way to explicitly disable it.

Such change will also benefit app-specific extra usecases: generic implementations won't try to do what wasn't intended.

[pjcozzi]

Sorry, I am confused. You think this will not be commonly done, but you suggest that we add it? If that is the case, we should seek more community feedback to avoid adding features that are not generally useful. If I'm misunderstanding, please let me know.

[lexaknyazev]

After re-reading what I've written, I agree it sounds confusing. But the point is very simple, actually:

Now, we have only linear interpolation.
I agree that supporting more complex interpolations is completely out of the scope, but why can't we have a standard way to disable it at all?

First, it will allow to do a "stop-motion" animation (including the simpliest camera cuts) or "slide-shows".
Now, asset has to double the number of keyframes to achieve that.

Second, it will make extending via sampler.extra more explicit: let's say we have a specific asset with a sampler.extra field. All viewers except asset's native app will do linear interpolation which may be completely irrelevant.

[pjcozzi]

Do you have real-world pragmatic use cases for the "stop-motion" animation? If not, do others? Perhaps in fields beyond games, e.g., robotics, data visualization, etc.? That can guide us to add this. Otherwise, I think this is a "solution without a problem" - and glTF painstakingly tries to avoid this; it is a format grounded in widely needed use cases.

[lexaknyazev]

Perhaps in fields beyond games

Even games can use it for "mechanical" animation look (think of second hand of a clock).

Btw, COLLADA recognizes the following interpolation types: LINEAR, BEZIER, CARDINAL, HERMITE, BSPLINE, and STEP.
All curve types (BEZIER, CARDINAL, HERMITE and BSPLINE) could be resampled to linear (with more keyframes) by converter/exporter. However, if animation consists of STEP interpolations we can benefit from keeping it intact to have smaller asset size and simplier runtime processing.

Thus, having LINEAR and STEP interpolations will make possible for glTF to cover all COLLADA cases at minimal cost.

[pjcozzi]

Constant is also in OpenGEX; see Page 77: http://opengex.org/OpenGEX.1.1.2.pdf

I suppose I am fine with adding it given that it is trivial, but I would really like to hear from one user who needs this before adding it to 1.1.

[pjcozzi]

Is this related to or perhaps even replaced by #1137?

[donmccurdy]

Is this related to or perhaps even replaced by #1137?

There are some differences in scope: #1137 deals with portable assets that can be dropped into a world and bring behaviors (animate on touch, etc.) with them. This issue revolves around complete scenes and assumes there is a concept of "scene time", so that's a very different use case.

My vote would be to only move forward on this if/when there are concrete use cases and interested parties driving it, and then to do so in an extension like #1137. No preference on whether we want to keep the issue open in the meantime.

[pjcozzi]

@donmccurdy OK with me.

@MarcoHutter up to you if you want to keep this open or close.

[javagl]

In order to prevent scattering the discussion between many issues, I think this one should be closed, as a more recent one has been opened with #1179