Skip to content

Plugin Development and API

Jump to bottom
Kaarlo Kananen edited this page Feb 22, 2016 · 9 revisions

Introduction:

Each plugin is implemented as a script placed in browser/plugins having the extension .plugin.js.

Individial plugins are registered with the system and placed in the editor context menu by adding them to plugins/plugins.json, which has the following syntax:

"Plugin Category": {
    "Plugin Name": "plugin_id",
    ...
}

plugin_id must match the filename (without .plugin.js) in browser/plugins

Plugin conceptual operation:

Each plugin can be roughly categorized into one of three groups:

  • Generators: Plugins that only have output slots. Represents sources of data, which can be anything from user input to execution context values.

  • Modulators: Plugins with both in- and outputs. Can be considered filters and represent operations on data from one or more sources.

  • Emitters: Plugins with only input slots. Usually provide final presentation of data which can be rendering, playback, recording or transmission to a remote receiver.

A few plugins have neither in- or outputs and these usually serve cosmetic purposes. One example is the Annotation plugin, which provides a persistent on-canvas note or comment.

Plugin implementation:

The beginning of each plugin implementation starts with the declaration of a constructor function and its assignment to a property my_plugin_id (which must match the filename without .plugin.js) in the plugins registry contained in the E2 namespace. The whole plugin declaration should be wrapped in a function() wrapper to avoid any variables leaking outside the scope.

It is recommended to inherit the prototype class Plugin

Example:

(function() {
MyPlugin = E2.plugins.my_plugin_id = function(core, node)
{
    Plugin.apply(this, arguments)

    // The default description and the slot desc attributes below are overridden by documentation
    // in /documentation/browser/plugins/plugin_name.md
    this.desc = 'Text string describing the plugin and its operation.'

    this.input_slots = [ 
        {
            name: 'in', // Slot display name. Mandatory.
            dt: core.datatypes.FLOAT, // Data type. Mandatory.
            desc: 'Default description of slot purpose to be shown in the info view.', // Optional
            def: 0
        }, 
        ...
    ]

    this.output_slots = [
        {
            name: 'out',
            dt: core.datatypes.BOOL,
            desc: 'True if in > 0'
        },
        ...
    ]

    this.state = {} // Optional state that is retained between Vizor sessions. 
}

// Inherit Plugin prototype
MyPlugin.prototype = Object.create(Plugin.prototype)

...

})()

The name of the property of E2.plugins the constructor is assigned to must match the filename of the plugin without extension, as well as the identifier used to register it in plugins/plugins.json.

In- and output slot arrays can be empty, but must be present. Slots declared at plugin creation time are termed static, as opposed to slots created at run-time, which are called dynamic.

The state member object is automatically persisted and deserialized by the Core. If a plugin does not require persistent state, the plugin.state member can be left undefined.

The plugin model is event driven. Event handlers are simply methods using reserved names that - if declared by a given plugin - are automatically called by the Core when required. Plugins authors can choose to implement any subset of these that are relevant for the behaviour of a given plugin.

Plugin methods:

Plugin.prototype.reset = function()

Called on plugin load, instantiation and when playback is stopped. If this method is omitted, it will not be scheduled for forced update after playback has been stopped and resumed. For the same reason, generator plugins will almost always implement this method. See also: stop().

Plugin.prototype.destroy = function()

If declared, this method will be called by the Core immediately before its parent Node is destroyed along with all associated resources.

Plugin.prototype.play = function()

Called immediately before graph playback begins.

Plugin.prototype.pause = function()

Called immediately after graph playback is paused.

Plugin.prototype.stop = function()

Called immediately after graph playback is stopped. Unlike reset, it will not be called as part of plugin initialisation or deserialisation.

Plugin.prototype.connection_changed = function(on, conn, slot)

Called when the state of a given in- or outbound connection involving the plugin changes.

  • on (boolean): True if a new connection was formed and false is an existing connection was deleted.
  • conn (connection instance): The object representing the connection that was just made ot is about to be destroyed. It has the following properties:
    • src_node (node instance): The source node of the connection. If the connection is outbound, this will be equivalent to the node parameter given to our construction function when the plugin is first instantiated.
    • dst_node (node instance): The destination node of the connection. If the connection is inbound, this will similarly be equivalent to the node parameter given to our construction function when the plugin is first instantiated.
    • src_slot (slot instance): The originating slot (see update_input() below for more details).
    • dst_slot (slot instance): The destination slot.
    • ui (connectionui instance): Only set when the plugin is on the currently active canvas and false otherwise.
  • slot (slot declaration): The slot of this plugin involved in the operation. Checking slot.type for equality with E2.slot_type.input or E2.slot_type.output can be used to determine whether the changed connection is in- or outbound.

This method is typically implemented when a plugin needs to respond to disconnection from inbound sources of data, but has other uses as well. Plugins that allow connection of any data type to be made to or from them, i.e. that declares slots (whether static or dynamic) of type ANY, make use of this functionality to adapt the data type of their own slots to that of the slot they're being connected to and to reset the data type of the slot back to ANY when a connection to the relevant slot is destroyed. See also: LinkedSlotGroup in the section below.

Plugin.prototype.update_input = function(slot, data)

Called whenever an inbound connection has new data to deliver. The Core guarantees that connected input slots are processed in the same order that they are declared by the plugin. No similar guarantee is made for processing of output slots.

  • slot: The slot that received the data
  • data: The new data value. This is guranteed to be of the correct type and match that of the slot, although not all datatypes are guaranteed to have a specified value. For all datatypes that are legally allowed to have no value such as Textures, an undefined value will always be null.

The slot parameter is an object containing the following members:

  • slot.dynamic (boolean): If set to true, indicates that this slot is a dynamic slot.
  • slot.desc (string): Slot description.
  • slot.dt core.datatypes reference.
  • slot.index (integer): Static slot index. Equivalent to the index of the corresponding slot declaration as specified in the constructor function of the plugin.
  • slot.is_connected (boolean): Indicates whether the slot is currently connected.
  • slot.name (string): The slot name as show in the UI.
  • slot.type (integer): The slot type. Either E2.slot_type.input or .output as appropriate.
  • slot.array: (boolean): true if data coming from this input is an array (of slot.dt type data), false otherwise
  • slot.uid (integer): Optional – only present if this is a dynamic slot.

To check which static slot is currently receiving input, testing for slot.name should be used instead of slot.index to avoid breaking graphs if inputs are added. The data can either be stored in plugin transient state by storing it in an arbitrary class property or be stored in the persisted plugin.state object, change UI state in response on incoming data and so on.

The default Plugin.prototype.update_input() implementation stores all input values into this.inputValues dictionary with slot.name as the key. It is recommended to override this behaviour only when needed.

If the plugin declares dynamic slots, slot.dynamic boolean can be used to differentiate between input to dynamic vs. static slots.

In general, no changes to plugin state should be done in update_input but instead the input values stored and any state changing done in update_state.

Plugin.prototype.update_state = function(updateContext)

Called once every frame after all calls to update_input() has completed, if:

  • One or more of the connected input slots have changed value.

  • This plugin has no output slots.

  • This plugin has no input slots.

  • This plugin is a nested graph and doesn't have its .always_update property set to false.

  • updateContext specifies the context which any temporal calculations should be executed in. It contains attributes updateContext.abs_t and updateContext.delta_t respectively for absolute time and delta time since the last frame.

Plugin.prototype.update_output = function(slot)

Called once for every connected output slot if update_state() was previously called this frame. Like update_input(), the slot parameter is the instance of the slot of this plugin being polled. See also: Plugin.prototype.query_output()

Plugin.prototype.query_output = function(slot)

If implemented, the Core will call this before calling update_output() for a given connected output slot. If this method return false, data flow will be blocked on that output connection.

Plugin.prototype.create_ui = function()

Called when the canvas on which the plugin resides is being switched to for editing.

jQuery is guaranteed to be available globally, so $ can be used, although excessive use of jQuery, especially for event handling is discouraged in production code for performance reasons.

Plugin implementers can create any nested set of DOM objects here, set up their own event handling and anything else they might like. When done, create_ui() is expected to return the root DOM element created, which will be dynamically shown on the surface of the plugin instance whenever visible in the editor. Always use the automatically persisted state plugin member to store UI state.

Plugin.prototype.state_changed = function(ui)

This method is called once after plugin creation or deserialization with ui set to null. This call can be used to do fundamental plugin instance initialization. If the plugin declares a UI, this method will be called seperately with ui equal to the root DOM element returned by create_ui() earlier. When this method is called the following is guaranteed by the core:

  • That the state member will be deserialised and available.
  • That the parent node will be fully deserialised with all data structures patched up and ready for use.

Other reserved plugin member names:

  • id (string): The plugin type name as declared in plugins/plugins.json.
  • updated (boolean): Flag indicating whether any input slots have updated this frame and whether update_state() need to be called as a consequence.
  • isGraph (boolean): A special state-flag used to identity nested graphs.
  • always_update (boolean): If this plugin is a nested graph, when this flag is set update_state() will be called each frame regardless of whether any input connections have changed provided this plugin is in itself residing in a graph that has ben updated this frame.

Core class interfaces:

Node

Node.prototype.get_disp_name = function()

Returns the string currently being used for the visible header of the node.

Node.prototype.add_slot = function(slot_type, def)

Adds a new dynamic slot to the current node.

  • slot_type: Either E2.slot_type.input or E2.slot_type.output.
  • def: Slot definition. An object equivalent of a static slot definition.
  • returns: A unique integer slot id.

Node.prototype.remove_slot = function(slot_type, suid)

Removes a dynamic slot from this node.

  • slot_type: Either E2.slot_type.input or E2.slot_type.output.
  • suid: Unique id of slot to remove.

Node.prototype.find_dynamic_slot = function(slot_type, suid)

Returns the slot definition for a given dynamic slot.

  • slot_type: Either E2.slot_type.input or E2.slot_type.output.
  • suid: Unique id of slot to remove.
  • returns: A slot definition or null if the slot could not be found.

Node.prototype.rename_slot = function(slot_type, suid, name)

Renames the specified slot.

  • slot_type: Either E2.slot_type.input or E2.slot_type.output.
  • suid: Unique id of slot to remove.
  • name: Desired new name for the specified slot.

Node.prototype.change_slot_datatype = function(slot_type, suid, dt)

Changes the data type of the specified slot. Unless the new data type is ANY, existing connection to or from the specified slot are destroyed.

  • slot_type: Either E2.slot_type.input or E2.slot_type.output.
  • suid: Unique id of slot to remove.
  • dt: Desired new data type for the specified slot.

Core

Core.get_default_value = function(dt)

Returns the default value for the supplied data type.

LinkedSlotGroup

This is a support class that can be used by plugins that declare multiple slots of type ANY and wants to link them such that if one of them are connected, all the controlled slots will match the datatype of the slot the initial connection is made with. Note that this is currently only supported for static slots. It is used in the following way:

(function() {
MyPlugin = E2.plugins.my_plugin_id = function(core, node)
{
    Plugin.apply(this, arguments)

    this.desc = '...'

    this.input_slots = [ 
        { name: 'in_0', dt: ..., desc: '...' },
        { name: 'in_1', dt: core.datatypes.ANY, desc: '...' } 
        { name: 'in_2', dt: core.datatypes.ANY, desc: '...' } 
    ]

    this.output_slots = [
        { name: 'out_0', dt: core.datatypes.ANY, desc: '...' } 
    ]

    this.lsg = new LinkedSlotGroup(core, node, [this.input_slots[1], this.input_slots[2]], [this.output_slots[0]])
    this.value = null
}

MyPlugin.prototype = Object.create(Plugin.prototype)

MyPlugin.prototype.connection_changed = function(on, conn, slot)
{
    if(this.lsg.connection_changed(on, conn, slot))
        this.value = this.lsg.core.get_default_value(this.lsg.dt)
}

MyPlugin.prototype.state_changed = function(ui)
{
    if(!ui)
        this.value = this.lsg.infer_dt() // Returns the default value for the inferred data type.
}
})()

Datatypes:

	this.datatypes = {
		FLOAT: { id: 0, name: 'Float' },
		SHADER: { id: 1, name: 'Shader' },
		TEXTURE: { id: 2, name: 'Texture' },
		COLOR: { id: 3, name: 'Color' },
		MATRIX: { id: 4, name: 'Matrix' },
		VECTOR: { id: 5, name: 'Vector' },
		CAMERA: { id: 6, name: 'Camera' },
		BOOL: { id: 7, name: 'Boolean' },
		ANY: { id: 8, name: 'Arbitrary' },
		MESH: { id: 9, name: 'Mesh' },
		AUDIO: { id: 10, name: 'Audio' },
		SCENE: { id: 11, name: 'Scene' },
		MATERIAL: { id: 12, name: 'Material' },
		LIGHT: { id: 13, name: 'Light' },
		DELEGATE: { id: 14, name: 'Delegate' },
		TEXT: { id: 15, name: 'Text' },
		VIDEO: { id: 16, name: 'Video' },
		ARRAY: { id: 17, name: 'Array' },
		OBJECT: { id: 18, name: 'Object' }
	};

Common pitfalls:

  • Performing computation in update_output(): Since output slots can be connected to more than once receiver concurrently, update_output() will be called once for each outbound connection that’s attached when the Core detects a successful run of update_state(). Thus calculations should always be performed in update_state() which will at most be run once per frame, and cached to be returned on request from update_output().

Advanced topics:

  • WebGL rendering: Vizor uses [https://github.com/mrdoob/three.js] (https://github.com/mrdoob/three.js) heavily for

  • Computing and caching output values in update_input(): As an exception to the above rule, it is possible is very simple cases to perform a calculation directly in update_input() and cache it for later emission by update_output(), omitting implementation of update_state() which will yield slightly better performance, although this approach should only be used sparingly.

For example, we can imagine a plugin which adds 1 to an input float value implement its update_input() and update_output() as follows:

MyPlugin.prototype.update_input = function(slot, data)
{
    // We have only one input, no need to mask on slot.index -- it will always be 0
    // 'data' is guaranteed to be a float.
    this.output_value = data + 1;
}

// No need for update_state() here...

MyPlugin.prototype.update_output = function(slot, data)
{
    return this.output_value;
}
  • Inhibiting normal data flow: The flow of data to any given input slot can be inhibited by decorating the slot declaration with a boolean flag with the name inactive set to true, i.e.: this.input_slots[index].inactive = true. Similarly, the inhibition can be revoked by removing the flag again, i.e. delete this.input_slots[index].inactive. Whenever update_state() is called, it’s implied that the plugin updated property is true. It is possible for a plugin implementation to abort data output based on logic in update_state()by setting updated to false, in which case no calls to update_output() or update_input() of the destination plugin will be made.
Clone this wiki locally