[Proposal] Consolidate chart-rendering to Vega-Lite #2385
Comments
|
@joshuarrrr Personally, I'm completely aligned on this recommendation. From my perspective, users don't particularly care about the underlying technology being used as along as they have a consistent, feature rich charting experience throughout. Given the integration and flexibility, Vega-Lite as the rendering layer and Vega-Lite as the declarative grammar to describe chart attributes and interactions seems like the best approach (versus the additional migration and re-writing with no direct customer benefit that something like Plot.ly would offer) seems like the best path to get to that. It also seem pretty clear, that Vega Lite is the only option that adds no additional dependencies, no incremental complexity, and no new maintenance bloat since it leverages existing OSD integration and implementation. In addition, I think their design principles (e.g., "Provide sensible defaults, but allow customization", "Favor composition over templates", and "Support gradual specification") and their development principles (e.g., "Strive to remain backwards compatible", "Generate generic Vega specifications", "Enable transition to Vega", "Fail gracefully") are very sound as an upstream dependency. Thanks for the research and recommendation! |
|
Accessibility support is a hard requirement. I see that there is a ticket open on Plot.ly's Github to invest, but it has been open since 2016. Are companies mitigating somehow? |
|
@brijos Accessibility of data visualizations is definitely important, and an area where there's room for improvement in most charting libraries and tools. The Vega-Lite discussion issue is a useful window into some of the technical challenges, as well as ideas for further improvements. Regardless of the charting library we choose, this is an area where we can lead and contribute a11y improvements upstream. |
|
[Triage] @joshuarrrr could you provide some insight on a migration path for existing charts. |
|
@anirudha - I know you have a lot of thoughts here and are a big proponent of plot.ly. Can you chime in? |
|
Priorities need to start working on an Abstraction interface , and the following requirements are met.
Decisions / Questions
in Summary,
|
|
The Observability plugin needs the following requirements reviewed as we migrate to the abstraction interface.
|
|
Thanks Ani! |
Do you mean we have to gather requirements from the plugins before making a proposal on any future abstraction layer? If so, I agree. I'm happy to help coordinate with the plugins with the help of @joshuarrrr and @ashwin-pc. Some of the work is already being down with the alerting / anomaly detection work @brijos is driving.
Can you help me understand a bit more here? Are you saying you want to see a list of visualizations prioritized for support in Vega-lite for consolidated chart-rendering? e.g., which will be included and which are on the roadmap? If so, I agree. @joshuarrrr I'm happy to contribute a first pass at this list in the eventual proposal.
I think we already have this in the form of GitHub issues. Both upstream in the charting library and downstream in OpenSearch Dashboards. We also would still have the same path for contribution of new features, overlays, etc. Do you agree? If not, what are the gaps you are seeing?
I think the ask here is to have a proposal on the data layer that supports the requirements of the abstraction layer, is that correct? I'm not sure these two need to come at the same time, but I'd be curious to hear your recommendation on order of operation. |
I think one of the goals of the abstraction layer would be to ensure that we have a path to enable all supported features in downstream visualizations as long as the abstraction layer is being used. Specifically, abstraction is used for a multitude of reasons (centralization, simplicity, improved testing, separation of policy and detail, etc.) but one of the goals of exposing an interface to handle visualizations/chart rendering in a unified way is to include all OpenSearch Dashboard supported features in all visualizations/charts. I realize it might be aspirational at this point, but I think it's the right goal to take. And we should aim high if we're going to propose a true abstraction layer.
I don't think multiple data sources, long-term community support/issues , governance model, ability to become maintainers, and performance aren't important. Specifically, this decision should not impact our ability to support multiple data sources, doesn't prevent us from becoming maintainers on any of the repos highlighted, or optimize for performance. I know this was just a statement, but are there any specific decision points that you think needs additional justification or exploration on? I realize some of these requirement will need to be considered in the abstraction proposal but is there any specific to the charting decision you don't think are supported above?
As far as I am concerned, legacy implementations do not affect the aspiration of OpenSearch Dashboard across features, user experience, long-term vision, etc. BUT we do consider implementation details when making implementation decisions ceteris paribus or when evaluating downstream impacts. Specifically, if we can achieve our user experience goal in multiple ways (ex. if any of the evaluated libraries could be made to work), we need to consider the path from the current state to a future state which requires understanding and evaluation of legacy implementations. Does that help?
I don't think this is true. There may not be substantial technical differences in the charting implementations in general, but there are meaningful technical differences in how charting exists in OpenSearch Dashboards in particular. For example, OpenSearch Dashboard already has a dependency on Vega-Lite, the Vega visualization feature is already implemented and already well used, Vega-lite visualizations can already be rendered in dashboards, and we already have expressions implementation for Vega-lite.
I appreciate the perspective here, but what is more promising in the long-term is very subjective. I think we should focus on building a solid experience based on what we need for our users now and in the future and make sure our abstraction layer is designed to facilitate interoperability long-term. The decision of which library can be revisited over time as long we keep our long-term goal of an abstraction layer that generalizes (i.e., removed from specific implementation requirements). Do you agree? Or do you see the not being an apache foundation a critical requirement to reconsider for charting libraries? |
|
[Planning] To make an epic and break down @joshuarrrr to make issues.
@joshuarrrr please communicate insight |
The migration path for existing charts is described and tracked in #2819 |
Is your feature request related to a problem? Please describe.
Today, there are at five different charting libraries that are use to render different visualization types in OpenSearch Dashboards (OSD). This makes it difficult for developers to provide a consistent user experience across visualization types, because any vertical feature that targets all visualizations must be implemented separately in each.
Essentially we want to
Describe the solution you'd like
Describe alternatives you've considered
Additional context
Why Vega-Lite?
Vega-Lite is built on years of academic research to create a concise, declarative JSON specification for interactive visualizations that enables users to rapidly create interactive visualizations. Even complex combinations of charts, layers, and interactions can be constructed by merging or updating a single simple object structure. It also makes it easy to write API wrappers, because they only need to generate JSON output (or partial output) following the Vega-Lite specification.
OSD already has a dependency on Vega-Lite, and it’s one we can’t easily remove even if we chose a different charting library (like Plotly.js). The fundamental value of the Vega visualization feature is that it allows end user’s to build and create totally new chart types by writing Vega/Vega-Lite specifications directly so there’s no sensible way to migrate that rendering to another library with different syntax.
The existing Vega visualization feature also provides a foundation for a low-code visualization creation flywheel. Users can use the built-in Vega-Lite editor to prototype new chart types not currently supported in OSD, and share these with the community, and it will be easy to “graduate” these chart types to Vis Builder or other easy-to-use charting tools. In the other direction, we can provide the Vega editor as an “advanced” editor to tweak or change the behavior of officially defined visualizations. This approach provides greater flexibility and power than existing systems which only allow custom JSON configurations without an interactive environment.
Vega-Lite visualizations have already been connected into all the important plumbing of OSD - they can be rendered in dashboards, have
expressionsimplementations, and even some styling to make them similar to the built-in visualization types, so much of the groundwork has already been laid.Some developers may initially find the Vega-Lite grammar to be a bit different from other JavaScript charting libraries or APIs, but that's part of its expressive power. The best way to get a sense of it is to look at examples.
To see Vega-Lite's approach to defining interactivity, see this example of a threshold based annotation/overlay with binding. (Note the use of
params, which is Vega-Lite’s system for setting and using variables within the declarative data structure.) See the gallery for other examples.To directly compare it to more traditional JavaScript charting libraries like Plotly.js, here’s a multi-series line chart with a unified tooltip and guide implemented in both:
How we decided
We did a quick survey of 20+ JavaScript charting library APIs, adoption, and usage to narrow-down our decision to four contenders: Vega-Lite, Vega, Plotly.js, and Apache Echarts, each of which met most of our criteria, summarized in the chart below (you may need to scroll right to see the entire table).
The text was updated successfully, but these errors were encountered: