Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add KHR_interactivity draft #2293

Draft
wants to merge 42 commits into
base: main
Choose a base branch
from
Draft

Add KHR_interactivity draft #2293

wants to merge 42 commits into from

Conversation

lexaknyazev
Copy link
Member

@lexaknyazev lexaknyazev commented May 31, 2023

@bhouston
Copy link
Contributor

Great work! What sections are next? Also what could we release as a draft extension for SIGGRAPH 2023?

|===
| Type | `math/pi` | Ratio of a circle's circumference to its diameter
| Output value sockets | `float value` | 3.141592653589793
|===
Copy link
Contributor

@aaronfranke aaronfranke Oct 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have Pi, we should also have Tau. https://tauday.com/tau-manifesto The value Tau is the circle constant (equal to 2*Pi, or rather, Pi is half of Tau). Using Tau usually results in more readable code. Tau is supported in many programming languages such as C#, Java, Python, GDScript, Rust, Unreal Blueprints, and more, so it's useful for interoperability with other languages, especially Unreal Blueprints which is conceptually similar.

===== Tau

[cols="1h,1,2"]
|===
| Type | `math/tau` | The circle constant, the circumference of the unit circle in radians.
| Output value sockets | `float value` | 6.2831853071795862
|===

Copy link
Contributor

@aaronfranke aaronfranke Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case more justification is needed, the KHR_interactivity document currently contains Degrees-To-Radians (math/rad) and Radians-To-Degrees (math/deg). These allow converting between degrees and radians. Tau can be used for a similarly important purpose, dividing or multiplying by this number allows converting between turns and radians, where 1 turn is τ radians. Tau can be used for more than just this, but the point is, if math/rad and math/deg are justified as a part of KHR_interactivity for converting angles, then math/tau is also justified for converting angles.

@lexaknyazev lexaknyazev force-pushed the interactivity branch 2 times, most recently from dc870c5 to 29e8acc Compare March 6, 2024 23:10
| `T value` | The custom variable value
|===

This node gets a custom variable value using the variable index provided by the `variable` configuration value. The type `T` is determined by the referenced variable. The variable index **MUST** be non-negative and less than the total number of custom variables, otherwise the node is invalid.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are variables accessed by index instead of by name? It would make more sense to me to set a variable called "money" than one called 27.

Comment on lines +2879 to +2882
{
"id": "variable",
"value": [0]
}

This comment was marked as resolved.


If the variable type is custom, the `value` property is defined by the extension defining the custom type.

== Nodes

This comment was marked as resolved.


If the pointer or the pointer template with all its substitutions applied can be resolved, the `value` output value is the resolved property value and the `isValid` output value is true.

If the pointer or the pointer template with all its substitutions applied cannot be resolved, the `value` output value is the default value for its type and the `isValid` output value is false.

This comment was marked as resolved.


Intermediate output progress values **MAY** be less than zero or greater than one.

==== Animation Control Nodes

This comment was marked as resolved.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closing this one as done, this will be in the text soon

@Hackn0214
Copy link

I apologize if this has already been mentioned somewhere.
I am currently checking the Interactivity Extension Specification.
How are p1 and p2 parameters used in node pointer/interpolate?

@hybridherbst
Copy link

hybridherbst commented Aug 26, 2024

A couple of practical questions based on my experiments:

  1. For looping an animation, is the expected approach to set endTime to positive infinity?

  2. For preventing animations overlap when multiple animations target the same node, is the expected approach to just stop all animations that might be playing, or to add an internal state of which animation was last played and then stop that?

  3. For interpolating rotations, how would the most-used quaternion interpolation – slerp – be implemented? I'm not sure a bezier curve (p1/p2) can be mapped to a spherical interpolation.
    Answer: quaternion interpolation is specified as using slerp already. The bezier curve is for the easing function for the t parameter going into the interpolation.

  4. Can the same output flow socket be connected to multiple input flow sockets, or must I add a flow/sequence node inbetween? For me that's not clear from the spec. I would expect that similar to values, I can just connect those outputs to multiple inputs and they're all triggered in the order they appear in the JSON.
    Answer: one output flow socket can only be connected to one input flow socket.

  5. I find flow/sequence to be confusingly named. It's not clear to me if "all output flows are activated one by one" means that each flow output first needs to finish before the next one is invoked, or that they are all fired immediately and just the order is specified here. For reference, other flow graph implementations that I'm aware of use sequence for "sequential" execution with waiting, and "parallel" for in-order parallel execution of the connected flows.

@warrenm
Copy link

warrenm commented Aug 26, 2024

  1. For interpolating rotations, how would the most-used quaternion interpolation – slerp – be implemented? I'm not sure a bezier curve (p1/p2) can be mapped to a spherical interpolation.

My understanding is that the control points specified by an interpolation node are to be used as the control points of a Bézier spline that is used as an easing function, à la cubic Bézier easing functions in CSS and elsewhere.

If that's correct, then the implementation would pass the normalized time parameter (on [0, 1]) to the easing function to get the eased progress fraction, then use that as the "time" parameter of the lower-level interpolation routine (whether lerp or slerp).

@hybridherbst
Copy link

Thanks! I found it now. The spec states that it's linear interpolation but also has this remark:

If the Object Model property is a quaternion, spherical linear interpolation expression SHOULD be used.

@Hackn0214
Copy link

Thank you for your response.
These parameters seem a bit challenging for designers to handle.

@Hackn0214
Copy link

Regarding the question I asked earlier, it seems it might have gotten buried and become hard to notice, so I’d like to ask again.
I couldn't find node/OnSelect(event/OnSelect) in the specification, but is it possible to handle events when an object is selected?

@lexaknyazev
Copy link
Member Author

I couldn't find node/OnSelect(event/OnSelect) in the specification, but is it possible to handle events when an object is selected?

This event is defined in the KHR_node_selectability extension, see #2422.

@Hackn0214
Copy link

I couldn't find node/OnSelect(event/OnSelect) in the specification, but is it possible to handle events when an object is selected?

This event is defined in the KHR_node_selectability extension, see #2422.

Oh, I thought this extension was an attribute to specify whether an object can be selected. I’ll read through the draft. Thanks!

@Hackn0214
Copy link

How would a connected graph with multiple output value sockets, as shown in the diagram, be represented in glTF?
gltfnode

@Hackn0214
Copy link

It's not a self-solution, but is there any issue with this kind of definition?

  "nodes": [
    {
      "type": "math/sign",
      "values": [
        {
          "id": "a",
          "node": 2,
          "socket": "1"
        }
      ]
    },
    {
      "type": "math/sign",
      "values": [
        {
          "id": "a",
          "node": 2,
          "socket": "0"
        }
      ]
    },
    {
      "type": "math/extract2",
      "values": [
        {
          "id": "a",
          "type": 1,
          "value": [
            0.0,
            0.0
          ]
        }
      ]
    }
  ],

@hybridherbst
Copy link

I believe this matches the output of https://github.com/KhronosGroup/glTF-InteractivityGraph-AuthoringTool/tree/initial-work-merge (minus the missing type declarations in your partial file).

@Hackn0214
Copy link

Thank you.
I'm comparing the behavior of my custom tool with the authoring tool as a reference.
By the way, is there a way to delete a node once it's placed on the graph in this tool? Currently, I'm restarting the tool to clear the graph.

@boguscoder
Copy link

@Hackn0214 backspace seems to remove just fine?

@Hackn0214
Copy link

@Hackn0214 backspace seems to remove just fine?
Ah... I was pressing the Delete key.

@Hackn0214
Copy link

When defining a custom event as shown below, if event/receive and event/send nodes refer to this custom event.

 "events": [
    {
      "id": "MyEvent",
      "values": [
        {
          "id": "Val_1",
          "type": 0
        },
        {
          "id": "Val_2",
          "type": 0
        }
      ]
    }
  ],

I think the generated node will have sockets as shown below, but what do you think?
gltfnode

@Hackn0214
Copy link

Is it possible to return the value pointed to by a pointer with the behavior graph?
For example, I would like to change the base color of a selected object's material.

/nodes/{nodeIdx}/mesh -> meshIdx
/meshes/{meshIdx}/primitibes/0/material -> materialIdx
/materials/{materialIdx}/pbrMetallicRoughness/baseColorFactor

@boguscoder
Copy link

If configuration value is array of length 2,3,4 or 16 it could be ambiguous if its vec2/vec3/vec4/vec4x4 or int[], this might not be a problem for JS but is a problem for strongly-typed implementations, is there any recommendation for them how to disambiguate?

@Hackn0214
Copy link

For example, how about specifying the type at the time of the call?
In fact, I would also like to have a feature that returns the number of elements as shown below.
/materials -> total material number in the scecne
/meshes/0/primitives -> Total primitive number

@boguscoder
Copy link

@Hackn0214 you already can get number of materials when using appropriate pointer , check out core pointers

@Hackn0214
Copy link

That's wonderful!
Thank you!

@Hackn0214
Copy link

I would appreciate it if you could add /meshes/{}/primitives.length

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.