Kedro-Viz Competitor Benchmark

[NeroOkwa]

Description

The goal of this research is to benchmark Kedro-Viz features with those of alternative/complementary tools (dbt, MLflow, Prefect, Great Expectation, Dagster, Metaflow, DVC) or those usually grouped in a similar category with Kedro (ML pipeline visualisation, orchestrators).

The emphasis would be on the visualisation/UI layer of these tools, towards identifying areas of improvement for Kedro-Viz.

Context

Why is this change important to you? How would you use it? How can it benefit other users?
The research would seek to understand:

1. What problem were they trying to solve for their users ?
2. How did they solve it and what was the effectiveness of this solution?
3. Is this a problem for kedro users? Have kedro users asked for this ? has it come up in user research?
4. How has Kedro tried to solve it ? was the solution effective ? is this a missing functionality for Kedro?

Possible Implementation

Research presentation, and prioritised list of features for improvement.

Outcome

Recording, Deck, Notes

Below is the prioritised list of features benchmarked with Kedro-Viz.

Feature	Is this on Kedro-Viz ?	Affects Kedro users?	Valuable to Kedro users ?	Will impact adoption ?	Decision	Type	Github link	Status
Metaflow - Card	Yes (with Vizro integrated with Kedro)	Yes	Yes	Yes	Yes (plugins)	Big bet	#1457
dbt - Filtering/syntax selection, Dagster graph filtering	Yes - in side panel but very limited	Yes	Yes	Yes	Yes	Big bet	#1784	Ongoing
Dagster - Re-materialise/backfill data from the graph - Run node/pipeline	No	Yes	Yes	Yes	Yes (ability to run node/pipeline, parameter editor from the UI)	Big bet	#1753 #1279
Dagster - Re-materialise/backfill data from the graph	Yes - autoreload flag (but not on UI)	Yes	Yes	Maybe	Yes (improve autoreload time and consider showing it in the flowchart)	Improve	#1751 #1704
Metaflow - Navigation bar/path specific URL	No	Yes	Yes	Yes	Yes	Improve	#1736	Ongoing
dbt - Docs + Lineage graph	No	Yes	Yes	Yes	Yes	Improve
Dagster - Tags and icons	Yes	Yes	Yes	No (engagement)	Yes (improve existing feature)	Improve	#1148
dbt - Color coding nodes	No	Yes	Yes. Only if combined with better filtering and layers	No (engagement)	Maybe	Improve
Dagster - Asset groups, expand/collapse all	Yes	Yes	Yes	Maybe	Maybe (improve existing feature)	Improve	#1752	Ongoing
DVC - Experiment tracking using GIT	No	Yes - 1 user for kedro-mlflow	Yes	Yes	No	N/A
Dagster - Left side bar and search field	Yes - sidebar Yes-search field	No / No	Yes / Yes (improving)	No/ No (engagement)	No	N/A

[astrojuanlu]

Moved this to https://github.com/kedro-org/kedro-viz/wiki/Competitors-benchmark-2024%E2%80%9002