S-D22.1 Meta-Modeling Infrastructure Y4M08

[KZzizzle]

22.1. Infrastructure work for meta-modeling: Y4M8

• robust parallel execution of pipelines
• scheduler (based on S-D25.4-5)
• result database-containers and extractors for parameterized results-equation framework (e.g., to handle non-linear parameter dependence, or correlated parameters)
• support for semi-cyclic pipelines, as required, e.g., for optimization and control (incl. necessary communication API)
• GUI components for meta-modeling

User story

• An iterator service is added to the workbench
• A parameter of type ‘real’ is added, it is named f, its distribution is chosen as linear, with a range for 5 to 10 in 11 steps
• As a result a corresponding output port (or attached parameter node) is created that can be connected to input ports of a pipeline. Initially, only pipelines exclusively consisting of computational services shall be supported.
• When the study is run, the multiple executions of the pipeline are started in parallel - with the parameter value changing 5, 5.5, 6, ... 10. Progress can be seen both on the workbench of the main study (check mark on a node means that all instances of that node have run, progress bar shows the average of all instances...), as well as on the list of simulation instances (see next point) (should we show the queue there?)
• A list of instances of the study can be displayed, which also shows the specific parameter values of that instance and its run status. The user can switch to a specific instance, upon which the iterator turns into a parameter value object.
• It shall not be possible to individually modify an instance (how about just adding a viewer?). To modify it, it must either be detached from the instances list, or copied and opened as a separate study (for both of these options, corresponding buttons shall exist - where? in the instances list? or within the instance after entering one?).
• Any output port of the pipeline in the main study does not stand for a single output, but rather for a list (database) of all the data at that output port in all the instances. Attaching a node to an output in the main study is equivalent to attaching that node to all instances and then executing it in all instances.
• If it is instead desired to perform an operation that makes use of one or multiple elements of that output list, a node of the type collector must be attached. A collector can be seen as the closing bracket where the iterator is an opening bracket. Any pipeline from a collector downstream only lives in the main study and is not duplicated in the instances (unless somebody is adventurous enough to parameterize part of that pipeline again...). Examples of collectors could be a service that performs statistics on the outputs, a service that computes the average of the outputs, a service that fits a function to the output, a service that extracts the result from a specific instance for further processing or visualization... For our user story, we will want to extract the output of a specific instance (f=6) for visualization by attaching an ‘extractor’ service to that output port.

Remarks

• It is the iterators that define the parameter combinations that should be run - not the instance list. For example, if one wants to edit the table of possible parameter combinations that an iterator generates, that should be done in that service, and not by editing the instances list. The instances list reflects the parameter combinations that have or are being run. At what point should the instances list be updated as a result of changes to the iterator objects?
• The scheduler should be clever enough to only execute parts of pipelines multiple times that change with changing parameter values
• How do we handle access to a specific instance through the API?

User stories beyond the Definition of Done:

• Iterators for non-floats (boolean, list of files...)
• Iterator for more than one parameter, with editable table of combinations
• Other examples of collectors (e.g., linear regression collector that performs a linear regression of a real-valued output as a function of a real-valued parameter
• Expressions: how do we implement functions of one or more parameters at the input of a service? Seperate calculator node, or named parameters and formulas. The latter is easier and more elegant and also readily applicable to s4l:web, but conflicts with the idea of parameters as nodes that are connected and the clean pipeline concept.

[pcrespov]

Update on sprint Chevrotain

Added

• snapshots (UI + backend)
• iterators (UI+definition in backend)

Ongoing

• complete version control (including branches)

Open

• auto commit iterations
• run sweeper

Details in ITISFoundation/osparc-simcore#2392

[pcrespov]

Update on sprint Capra delle nevi

Completed

• snapshots (UI + backend)
• iterators (UI + definition in backend)
• complete version control (including branches)

Ongoing

• auto commit iterations
• run sweeper

Open

• interconnecting snapshots

Details in ITISFoundation/osparc-simcore#2392

[pcrespov]

Update on sprint Meerkat

Completed

• snapshots (UI + backend)
• iterators (UI + definition in backend)
• complete version control (including branches)
• auto commit iterations
• run sweeper
• interconnecting snapshots
• retrieving results from database and parameters

Therefore, from a feature point of view, this case is RESOLVED

Next steps

• Last PR peer reviews in the coming days before it hits master branch
• Release to testers ( and polish details)
• The infrastructure (i.e. this case) is completed, the next steps are
  completing use cases of sensitivity S-D22.2 Meta-Modeling sensitivity analysis Y4M08 #353 and uncertainty S-D22.3 Meta-Modeling uncertainty assessment Y4M10 #354 analysis.