Supporting the reporting use case on Kedro-Viz

[yetudada]

Description

This ticket describes enabling users to do reporting using Kedro-Viz. This issue proposes the following user journey; a user would be able to:

1. Update parameters for a pipeline run
2. Do a pipeline run with the updated parameters
3. See output charts (already possible with Plotly and Matplotlib integration; could be extended with VizX integration)

The focus of this discussion is steps 1 and 2. This is an unvalidated bet that we can increase the usage of Kedro-Viz if we better support reporting.

Why should we explore something like this?

There is qualitative evidence to support the build of functionality like this. The bet is that this functionality, coupled with #1116, will make Kedro-Viz a single-entry point for visualising outputs from a run and will also support non-technical users who need the insights, resulting in increased usage of Kedro-Viz.

Here is some qualitative evidence to suggest that this is needed:

• ModelUI [43:17] and the VizX extensions [38:04] were hackathons ideas from QuantumHack 3.0
• It was discovered that a user has built reporting Jupyter notebooks with notes on which parameters the user could change, use of Kedro's API to do a pipeline run in the notebook and support for output charts in the Databricks research [08:20]
• A separate user query: "In our project, we would like each model to be summarised by a detailed report (with elements like many different graphs, tables with parts of data, variables distribution, variable importance graphs, performance metrics, pieces of parametrised text etc.). Probably optimal format of such a report would be HTML or PDF. Would you be able to suggest what would be the best way to create such reports with Kedro? Currently, we are developing nodes that write a report to file, skipping the Kedro output system, but this is quite messy (this node is huge). We are also considering using Jupyter notebook, which rerun will be triggered by Kedro pipeline after the model is retrained and saved to HTML (again within the Kedro pipeline). Still, this way report would look quite crude. Please share your experiences and suggestions on how to approach best this type of reporting in Kedro."

What's next?

We would need to validate that this is a direction that we should go in by sourcing more user input. This design is straight out of my dreams and is therefore not accurate to full user requirements nor is it technically feasible.

[datajoely]

Perhaps the right way to contextualise this around a "Run experiments" workflow and make the tracked parameters the pieces that one can change?

[datajoely]

In general I think there is a great opportunity for Kedro to empower our Python engineers without front-end skills to make UIs. Exposing Kedro library components the catalog, session and runner in a way that allows users to build interfaces that are hosted as a panel in Viz would be incredible.

[yetudada]

Additional things to consider from @NeroOkwa: "Understand VizX solution to reporting use case."