[FEATURE] Integration: align datasource with workflow

[YANG-DB]

Is your feature request related to a problem?
Integration currently can contain multiple datasources - each one may have one or more workflows.

Some datasource cant use workflows that are relevant only for other datasources such as:

• OpenSearch datasource cant use the queries workflow
• S3 datasource cant use the non MV dashboard workflow

On a general note we sould also allow workflows to mutual exclusive such as when different alternatives are possible for creating an MV:

• Live MV
• Summarize MV

What solution would you like?
We need to add meta-data to associate the workflow with its datasource - and as a result only show the workflows that correspond with the selected datasource

Do you have any additional context?

[Swiddis]

For this I was thinking of maybe adding an applicable_data_sources field to the workflow definition

interface IntegrationWorkflow {
  name: string;
  label: string;
  description: string;
  enabled_by_default: boolean;
  applicable_data_sources?: string[]; // default: all
}

Then we could see workflows specified as

{
  "name": "flint-live-dashboards",
  "label": "Dashboards & Visualizations For Flint Integrations using live queries",
  "description": "Dashboards and visualizations aligned with Flint S3 datasource ",
  "enabled_by_default": false,
  "applicable_data_sources": ["s3"]
},

And when rendering workflow options, we filter based on the active connection type

const cards = integration.workflows
  .filter((workflow) => workflow.applicable_data_sources
    ? workflow.applicable_data_sources.includes(config.connectionType)
    : true
  )
  .map((workflow) => {
    return (
      <EuiCheckableCard
        id={`workflow-checkbox-${workflow.name}`}
        key={workflow.name}
        label={workflow.label}
        checkableType="checkbox"
        value={workflow.name}
        checked={useWorkflows.get(workflow.name)}
        onChange={() => toggleWorkflow(workflow.name)}
      >
        {workflow.description}
      </EuiCheckableCard>
    );
  });

This would allow optionally specifying additional filters for workflows while preserving current behavior for all unmodified integrations, and also moves toward activating workflows for OS integrations in #1519.

It's been on my to-do list for a while, but no bandwidth -- open to a PR here.

[Swiddis]

Mutual exclusivity is also something I've thought about but I don't know if I have a great solution. A minimum fix would likely be to have a excludes: string[] field that excludes other workflows when that workflow is selected, but I'm concerned about how things would look if we start needing more complex exclusion-inclusion constraints. More robust would be to have some sort of expression field that lets you write arbitrary workflow subset validation expressions, but that's a solution that's way too interesting to actually be good.

Maybe I could also think about how to formalize some sort of install picker directive that would look something like:

"workflow_selector_requirements": [
  { "one_of": [ "table_a", "table_b" ] },
  { "at_least_one": [ "dashboards_a", "dashboards_b" ] },
  { "at_most_one": [ "extra_a", "extra_b" ] }
]

It's not too bad to implement validation-wise, but getting the UI to comprehensibly handle issues would be a challenge.

I'm inclined to say let's split mutual exclusivity into its own issue, since data source alignment is considerably more straightforward.