New Epic: Simplify config

[jmcook1186]

Here's another upcoming change - I'm signalling it in advance here so you all have time to comment and give feedback before it gets worked up as a ticket in one of our development sprints. Take a read through and let us know your thoughts!

Background

IF has been through several iterations. Originally we just aimed to calculate SCI scores for simple architectures, now its something much more general purpose and more akin to a low level protocol for environmental impact calculations. This means we have redesigned the way we pass config data around IF a couple of times too. We currently support three ways to get data into a plugin:

• input data
• node-level config
• global config

Node level config was intended to reconfigure already-initialized plugins so that one instance can be re-used in multiple parts of the tree. This was useful before we migrated most of our pipelines over to generics. Now, node-level config is rarely used. Instead, we create named instances of our plugins that are dropped into each pipeline as needed. This is much easier to understand and audit, even though it makes the initialize block more verbose. For this reason, node level config is rarely used nowadays, and is causing some confusion for plugin developers and builders. We think it's time to remove node level config as a concept from IF.
In this case, all config for a plugin will be set in the global config section in the initialize block, and all input data will come from inputs.

Its a lot easier and clearer to configure multiple instances of a plugin in the initialize section and then use specifically named instances of plugins in the component pipeline. For example:

initialize:
  plugins:
    teads-curve-spline:
      path: "@grnsft/if-unofficial-plugins"
      method: TeadsCurve
      global-config:
        interpolation: spline
    teads-curve-linear:
      path: "@grnsft/if-unofficial-plugins"
      method: TeadsCurve
      global-config:
        interpolation: linear
.
.
.
tree:
  pipeline:
    - teads-curve-spline
    - teads-curve-linear
  inputs:        
                

Problem statement

Having multiple places to configure a plugin is confusing, it’s unclear for plugin developers if they should architect for multiple instances of a plugin with different global config options, or single-instance of a plugin with multiple node level configs.

Node level config is difficult to rationalize, node-level config applies to multiple levels of a tree and can be overridden by child component node-level config.

From the hackathon, it looks like no plugin authors used node-level config, so it’s not a feature which is broadly adopted - perhaps because we do not use node-level config in most built-ins, showing that even the core team don’t get much value from it.

Proposed Solution

Remove node-level config from the interface, review all our existing plugins and move everything to global config if there is still config at the node-level.

Tasks

1. Audit all our builtins t identify all the cases where node-level config is used
2. Refactor any builtins that use node-level config so that all config comes from global scope
3. Refactor IF so that any logic related to node-level config is removed
4. Update documentation to remove any mentions of node-level config
5. Write update-guide for plugins builders that need to refactor our node level config

How you can help

You can read through this post and give feedback in comments, especially if you are a plugin developer that currently relies on node-level config. Later, when the specific tasks are available as tickets on our issue board you can let us know if you want to work on one. There may be some that are reserved for core developers, but in general we are keen to open up IF development to the community.

@jawache @zanete @narekhovhannisyan @MariamKhalatova @manushak