Support for annotations across Kibana

[alexfrancoeur]

We recently added the ability to add annotations to your machine learning data. It has been a longstanding request in Graph to add user annotations (#17914), we support viewing annotations in the Time Series Visual Builder and users are looking to edit / add fields in to a document in Discover (#36556). We have had many discussions around adding annotations in a consistent fashion across Kibana so that users can explain anomalies in their data and provide details around ongoing and old investigations / incidents.

As we think of all the areas annotations can be useful to the applications in Kibana, it probably makes sense to introduce as a commonly used service first. Then we can decide UI/UX flows, how to visualize in Elastic Charts and CRUD UI's.

While we don't have immediate plans to build out this feature and the requirements will likely evolve, we'd love to hear some details around how the community would utilize such a feature. Please feel free to share your thoughts.

cc: @AlonaNadler

[elasticmachine]

Pinging @elastic/kibana-app

[Paul-de-Barbeyrac]

Hello,

Thanks for this feature. We would love to see it available in the dashboards looking at the time series vizualisations.

In our organization, we monitor industrial equipments (Equipments->PLC->KEP Server->RabbitMQ->Logstash->Elastic->Kibana) in real time. Operators and supervisors on site take actions based on the behavior monitored and seek for support from machine experts at HQ.

Making the annotation feature available in the dashboard / time series graph would be awesome since the end users don't use the ML learning module. Yet, it would be very valuable if they were able to input some comments straight from their space on the graph fluctuations as they take actions on site. It would let all the support teams be aware of what is going on in the field, what initiatives were taken by operators as first troubleshooting steps and adivse best on what to do next.

That would also be useful on the long term if Kibana deploys some sort of supervised learning.

Hope it helps,
Thanks to the helpful community,

Paul

[peterschretlen]

This has come up in the context of alerts, which are often overlaid on charts as annotations. These help to both visualize the alerts and correlate them with other metrics or events.

[dgieselaar]

There's a need for annotations for observability solutions as well. We're currently planning on building some kind of API that works for observability solutions, but it would be great if we can somehow align/make that compatible with a Kibana-wide annotations effort.

[alaudazzi]

A feedback mechanism was recently activated on the Cloud doc pages, and will be soon extended to the rest of the Elastic documentation.
Writers are going to monitor the feedback through a Kibana dashboard. It would be helpful to add annotations to triage and categorize the feedback responses (for ex. as wish, bug, as-is, etc.)

[ymao1]

@alexfrancoeur What is the expectation for Alerting in regards to annotations support?

[alexfrancoeur]

I think I added the label as a result of Peter's comment above (#36628 (comment)). Generally, I think there are a few different needs for annotations across the board.

There are end user annotations on the data, which I think this issue was originally opened for. And then there are system generated annotations. We can think of this as the annotations Observability creates from APM agent events, but we can also think of this in terms of alerts as data. If alert history has a standardized schema, we can begin to imagine a more holistic experience across Kibana for overlaying them in charts, similar to this example.

If there is an open issue that encompasses alerts as data, we can probably remove the alerting team @ymao1. This meta issue would likely need to be broken out into a variety of things. An implementation for creating user generated annotations, how and where we consume them, schema, permissions, etc. This is a large task, but one still on the backlog!

cc: @VijayDoshi @arisonl