[Meta] Content Management - Providing context, understanding activity and adoption

[alexfrancoeur]

Description

The content management initiative focuses on improving the overall experience for governance, management and discoverability within Kibana. This project definition is focused on high impact low hanging fruit and will be mostly leveraging the saved object metadata that exists today. In this first iteration, we’d like to surface all useful metadata in order to provide context, understand activity and adoption. This is the first bucket of work of many for this initiative and is the public output of an internal project spec.

Related issues

• Add date created, date modified, other metadata to Saved Objects · Issue #9202 · elastic/kibana · GitHub
• [Visualize] Ability to filter visualisations by metadata (author, created date, modified date etc) · Issue #84227 · elastic/kibana · GitHub
• [Saved Objects] Report usage and last used date #125795
• Kibana usage information · Issue #81130
• Google Analytics on Kibana #5580
• Add tags as part of the URL for filtering objects on Analytics listing pages #132358

Stakeholders

Roles	Contacts
Responsible / Driver	@alexfrancoeur
Accountable / Approver	@alexfrancoeur @teresaalvarezsoler @ghudgins
Contributor	@sebelga @cchaos @ryankeairns
Informed	@clintandrewhall @rayafratkina @arisonl @lukeelmers @thomasneirynck @majagrubic @rshen91 @crob611 @sixstringcode @stacey-gammon

Target personas

Persona	Use case
Administrator	As an administrator I’d like understand which dashboards are actively being used and viewed for migration and cleanup purposes
Author	As an author, I’d like to easily provide context to analysts leveraging my content As an author, I’d like to understand which content is being adopted
Analyst	As an analyst, I want to view the most recently updated content As an analyst, I want to view the most popular content As an analyst, I want full context to completely understand the content I’m looking at
Solution user	As a solution user, I want to view the most popular integration content related to the technologies I care about As a solution user, I want to view the most recently updated content As a solution user, I want to view the most popular content As a solution user, I want full context to completely understand the content I’m looking at

Acceptance Criteria / Definition of Done

Objectives

• Introduce new view count metadata for content
• Surface all available useful metadata at the content detail view (individual dashboard, visualization, saved search, saved object management, etc.)
• Surface all available useful metadata at the content list view (browse views for dashboard, visualization, saved search, saved object management, etc.)
• Optimize content list views for discovery based on use case for viewing the content (what’s new, what’s most popular)
• Reduce time to value for content navigation
• Partner on one solution use case to bring this experience to a solution (example: cases, infrastructure inventory views, rules, timeline, etc.)

Supporting telemetry / measurements

• View count telemetry to better understand visibility patterns of content in deployments
• Count of times content list views are sorted by recency and popularity
• Measure duration from landing on content list view to viewing content before and after content changes

Capabilities and requirements

Below are both the functional and non-functional requirements for this initial work focused on content management.

Functional requirements

ID	Requirement
1	[Dashboard | Visualization Library | Saved search | Save Object Management] list view should add updated_at as a sortable column
2	[Dashboard | Visualization Library | Saved search] should add new metadata for total_views and 30_day_views that count the number of times they have been accessed
3	[Dashboard | Visualization Library | Saved search | Save Object Management] list view should add total_views and 30_day_views as a sortable column
4	total_views and 30_day_views should be available programmatically through an API for Elastic administrators looking to automate tooling
5	[Dashboard | Visualization Library | Saved search | Save Object Management] list view should provide both ascending and descending alphabetical, last updated, most viewed (total) and most viewed (30 days) sort options
6	[Dashboard | Visualization Library | Saved search | Save Object Management] list view should have a tags column and easily filterable by tags
7	[Dashboard | Visualization Library | Saved search | Save Object Management] list view should not have description as a sortable column but instead showcase as additional metadata
8	[Dashboard | Visualization Library | Saved search | Save Object Management] list view should make relationships (references) flyout readily available
9	[Dashboard | Visualization Library | Saved search ] details view should have all available metadata easily accessible.
10	When applicable, the metadata should be actionable (tags can navigate back to list view filtered on that tag for example) in the [Dashboard | Visualization Library | Saved search] details view.
11	Telemetry collection will be required to measure success criteria
12	[Dashboard
13	Views / popularity should be configurable by the administrator. It should be on by default but disableable and should have the ability to reset counters
14	When re-visiting a list view, users should easily be able to jump back into prior content

Technical and performance requirements

ID	Requirement
1	updated_at will need to be validated that it is updated every time a saved object is saved
2	Collection of total_views and 30_day_views should not impact performance or stability of Kibana
3	Telemetry collection on content management capabilities should not impact performance or stability of Kibana

Milestones / phases

Milestones and phases still need to be defined in more detail, but here's an initial outline that is subject to change.

• Product Research
• Finalize UX & designs for list and details views
• Technical research
• Implementation plan

Possible implementation milestones (to be confirmed)

• [Content management] Surface "updated_at" in TableListView #132134
• Extract or create new list view as a reusable component for [Dashboard | Visualization Library | Saved search | Saved Object Management | Other]
• Augment saved objects to include view count
  • RFC to propose a solution to the core team (or app service team)
  • POC for possible solution ([POC] SO Metadata events streams #130976)
• Implement new component in Dashboard list view
• Add metadata to Dashboard detail view
• Rollout to subsequent analytics views
• Audit and engage with solution teams for adoption where applicable

Design

We are in need of some design help here, even with the limited scope of this work. The UX should focus on providing context and optimizing for discoverability based on recency or popularity. Some "PM mockups" and earlier designs can be found below. @ryankeairns @cchaos, we could really use your help here.

List view and discoverability enhancements

Details view and contextual metadata

Really old mockup of a “grid” view that’s still (mostly) relevant

Open Questions

• @alexfrancoeur What is the level of effort and dependency on the core team to provide view count metadata for select SO’s?
  • A: No core changes to SO’s are required to add views to a saved object. We should be consistent though and if we need to standardize on a schema for certain fields, we’ll need to engage the core team.
• @alexfrancoeur What use case is the best one to get started with? Assuming dashboards.
  • A: Yeah, it was dashboards
• @alexfrancoeur What solution use case is a good one to get started with after the initial analytics one?
• • A: Cases or saved views in o11y / security
• @alexfrancoeur In a meeting earlier today, @rayafratkina had raised the topic of a single content view / library on the analytics overview page vs. improving individual content list pages. Is that the direction we’re heading?
• A: Most likely yes, but only building foundational elements for it in this phase.
• @teresaalvarezsoler Do we know how users use tags right now? It could be interesting to give us an idea on what the users value most to make dashboards filterable and discoverable
  • A: ~50% of clusters with > 100 dashboards use tags, ~75% of clusters with > 1000 visualizations use tags.
• @lukeelmers What is a view, actually? Does a GET request count? Do API only requests count?
• @cchaos Can we make a list of which metadata we feel is important to surface within the Listing pages (discovery/explore) vs Management.
  • A: @alexfrancoeur For this phase we are talking about only adding two types of metadata for both discovery and management use cases. Last updated and popularity / views (timeboxed, total timeframe). I would view both of these as equally useful for discovery and management use cases.
• @cchaos What sorting capabilities could we use today?
  • A: @cchaos Alpha: A-Z, Z-A, Last updated: Recent first, Oldest first
• @cchaos Can you explain a workflow where this (relationships) would be pertinent to see inside of a Dashboard listing page vs within Management?
  • A: @alexfrancoeur The use cases I saw associated with this are primarily related to trusting the data and understanding its lineage. While this is very useful for administrators, I think you could argue it’s equally important for establishing trust and providing context.
  • What dataviews or saved searches are associated with this dashboard (or other SO)?
  • What spaces is this saved object in?
  • Are any of these visualizations in the library?
  • Who created the referenced objects? (future)
• @rayafratkina Should we use a different name than saved objects for this content?

[alexfrancoeur]

Added two more capabilities to the requirements based on the kickoff discussion earlier this week:

• [Dashboard | Visualization Library | Saved search ] list view sort setting should be persisted with a user. When I leave and come back, they should look the same
• Views / popularity should be configurable by the administrator. It should be on by default but disableable and should have the ability to reset counters

Added open questions to the meta issue with one specifically from the kickoff meeting

• What is a view, actually? Does a GET request count? Do API only requests count?