[Dashboard] Redesign Add Panel Experience

[nickpeihl]

Problem

As we add more panel types to dashboards, the current popover gets small and users need to scroll to find the options at the bottom making it harder to find the option that users are looking for.

Solution

We need a experience that scales as the number of panels grow and provides a better organization for the options so that users can find their panels more easily.

Requirements

• When users click in the Add panel button, a flyout will be open and similar panel types will be grouped and searchable by name.
• The section Machine Learning & Analytics will only appear if the has a >platinum license
• The Observability options should NOT appear if the user opens a dashboard in Search or Security projects (new left navigation)

The Aggregation based item should trigger the following modal:

Note these suggested changes in the modal compared to the existing modal:

(a) a more specific title (should we say Legacy in the title somehow?)
(b) remove the “Select a different visualization link”. Closing the modal should be enough. The flyout should remain visible after closing the modal
(c) remove that image from the background, afaik that is an outdated UI detail that we don’t use often anymore.

Figma file

[elasticmachine]

Pinging @elastic/kibana-presentation (Team:Presentation)

[ThomThomson]

This is looking awesome!

I'm wondering if we should remove the toolbar entirely to give that space back to the dashboard contents, and use a top nav item instead for the add panel button.

A couple other benefits of this approach off the top of my head are:

• The user can still access the add panel button, even when the page is scrolled down.
• It fits in nicely with the other actions that work on the whole dashboard, and is a natural place for dashboard authors to go to add content.

@andreadelrio, I know in your original designs you had a floating button on the bottom right of the dashboard. Is that still a design you're interested in using?

[Heenawter]

++ for getting rid of the toolbar completely and finding another place for the button - either in the top nav bar or as a floating action (although floating actions tend to have issues with a11y, which is something we should be mindful of). It makes dashboards look so much cleaner (•̀ᴗ•́)و

[nreese]

You know whats funny is this is moving back towards the design from Kibana 7.x. For example, here is some screen shots from 7.9. I always liked this menu better then the current toolbar. We should chat with design and understand why toolbar was added in the first place before removing it. I think we need to understand what problems it was trying to solve.

Where clicking "Create new" opens

[nreese]

Toolbar added in 7.11, #83342

#82906 provides an interesting read on why we moved to toolbar in the first place

[nreese]

cc @ryankeairns

Maybe you can provide some more background on dashboard toolbar and the problems it was trying to solve?

[ThomThomson]

Thanks for finding these @nreese! You're right that we're moving in a direction that is more in line with pre 7.11 Dashboard. I believe that we can get the best of both worlds with this new design, especially if we take into consideration all of the reasons we moved to a toolbar in the first place!

@ryankeairns would definitely know more than me, but from my recollection, the reasons that we created the toolbar are:

Why did we create the toolbar

1. Highlighting Lens in the hopes of driving more users there. As far as I remember, the go to lens button in the right pane of the new visualization window pre 7.11 was hardly ever clicked, as it wasn't in the main flow of content. A button that allowed the user to get to Lens in one click was extremely important to product at the time.
2. Showcasing other editor types in a standard and always visible way so that dashboard authors can easily access them with one click.
3. aligning the save menu actions on the Dashboard and the editors (Lens, Maps etc) to be in the emphasized, rightmost spot. We were big on aligning the UI between dashboard and the editors at that point, and having create new as the most emphasized menu item.

How this new design could address these items

1. In terms of highlighting Lens, this new design does introduce one more click, but the annoyance of that will hopefully be offset by the fact that we allow authors to select the chart type (donut, bar, line etc) right from the add flyout. The way that all of the different Lens subtypes are listed out also plays double duty by making Lens seem like the default option.
2. I think that highlighting other editor types is the genesis of this new design. @andreadelrio correct me if I'm wrong but in the usability tests for Tabs run by product & design, users almost always went to the create visualization button first, expecting tabs to be a subset of visualization. This was despite the fact that the create tab option / icon existed in plain view a few icons to the right on the toolbar. It was almost too good at driving users to Lens, to the detriment of all other types. This new design does a better job highlighting all of the available types along one axis where we can use the order / number of items, and things like new or deprecated tags to encourage dashboard authors to use certain editors.
3. For aligning the save menu actions, as long as we're thoughtful about how we use the top nav bar, we should be able to keep Dashboards and the editors aligned in this way. My initial design has the Add panel button one to the left of the save button.

There are absolutely reasons for the toolbar that I've forgotten or misrepresented! We should keep the discussion open to make sure that this design really does fit the best of both worlds.

[andreadelrio]

I'm wondering if we should remove the toolbar entirely to give that space back to the dashboard contents, and use a top nav item instead for the add panel button.

A couple other benefits of this approach off the top of my head are:

• The user can still access the add panel button, even when the page is scrolled down.
• It fits in nicely with the other actions that work on the whole dashboard, and is a natural place for dashboard authors to go to add content.

@andreadelrio, I know in your original designs you had a floating button on the bottom right of the dashboard. Is that still a design you're interested in using?

I like the idea of getting rid of the toolbar altogether. Using the top nav to house the + Add (panel) button would work well though we'd need to define what we want the primary action to be (Save or Add?). The floating button idea was something I came up with because I didn't think we could fit that button in the top section of the application. However, seeing the image above (and screens from previous versions of Kibana that Nathan shared) I think this works just fine. We could potentially show the floating button when there's scrolling as a nice-to-have.

[ThomThomson]

@andreadelrio the top nav actually scrolls with the page!

[andreadelrio]

++ thanks Nathan for bringing this up.

How this new design could address these items

1. In terms of highlighting Lens, this new design does introduce one more click, but the annoyance of that will hopefully be offset by the fact that we allow authors to select the chart type (donut, bar, line etc) right from the add flyout. The way that all of the different Lens subtypes are listed out also plays double duty by making Lens seem like the default option.

One of the motivations for this proposal is actually to bring a little bit more of Lens into Dashboard to drive its adoption. We do this by directly presenting Lens' viz types to users. Also, we found that the word Lens itself (as included in the Select type dropdown) doesn't say much to new users.

2. I think that highlighting other editor types is the genesis of this new design. @andreadelrio correct me if I'm wrong but in the usability tests for Tabs run by product & design, users almost always went to the create visualization button first, expecting tabs to be a subset of visualization. This was despite the fact that the create tab option / icon existed in plain view a few icons to the right on the toolbar. It was almost too good at driving users to Lens, to the detriment of all other types. This new design does a better job highlighting all of the available types along one axis where we can use the order / number of items, and things like new or deprecated tags to encourage dashboard authors to use certain editors.

++ I would also mention other "content" types. As we add more content types to Dashboard (think Controls) and in the future (panels, tabs and/or navigation embeddable) the toolbar starts suffering from growing pains and doesn't seem to scale that well (we found this in usability testing sessions). One of the main advantages of using the flyout to present all editor + content types available is that we can organize them in sections guiding users so they can more easily find them and figure out which one they need.

I will bug Ryan to get his two cents here as he seems to have a lot of the background.

[nreese]

One concern is "+ Add panel" button does not stand out at all and gets lost in the menu. I think "+ Add panel" button needs to be as strong or stronger call to action then the save button.

This was a big driver of the toolbar and is reflected in #82906

[ThomThomson]

That's a really good point. Cause when you're editing a dashboard, adding content to it is the thing you'd want to do, but saving is up there in importance as well.

I wonder if we should introduce some more configuration options to the top nav menu, because it currently has only two levels of emphasize, so either the save button or the Add panel could get lost in the fray if it was just as emphasized as the options or save as button for instance.

Maybe we need a design where whichever button is second in importance uses the secondary action button style from EUI. I don't have a strong opinion on which button should be primary and which should be secondary.

[andreadelrio]

That's a really good point. Cause when you're editing a dashboard, adding content to it is the thing you'd want to do, but saving is up there in importance as well.

I wonder if we should introduce some more configuration options to the top nav menu, because it currently has only two levels of emphasize, so either the save button or the Add panel could get lost in the fray if it was just as emphasized as the options or save as button for instance.

Maybe we need a design where whichever button is second in importance uses the secondary action button style from EUI. I don't have a strong opinion on which button should be primary and which should be secondary.

To keep things consistent with other applications we could keep the Save button closest to the right with a fill and put Add to its left with a more subtle fill.

[ryankeairns]

This is fascinating to revisit.

It seems most of the reasons/motivations are surfacing:

• Lens should be the default editor of choice
• It should be quick (read: minimal actions) to add a new visualization
• There were plans for native Dashboard elements (e.g. Markdown/text, image, etc) that would be 'quick additions' via the toolbar
• Historically, the distinction between view and edit mode was not apparent - this is a key item to keep in mind. That said, we later added the badge, which helps. Having the toolbar occupy a row makes a very noticeable difference in the two states; downside being it takes up real estate.
• Add vs Create was tricky once we introduced the library. We weren't desiring to promote the library (i.e. Add implies this), per se, rather it was a side effect / the 'old way' of doing things vs by value (i.e. Create)

Regardless of which path you all head down, I would highly recommend testing out the solution(s). There are important nuances that may or may not promote the behavior we desire such as driving people to Lens.

Other considerations:

• How does the new solution impact portable/embedded dashbaords?
• Supposing a world where we only have Lens, Maps, and Vega... how does that impact the solution?
• Reminder: Canvas uses this toolbar, too
• There is funkiness with new panels being appended to the bottom of a dashboard. Something else to keep in mind as you think through the workflow.

I do like/see the potential of surfacing the viz type selector from Lens as the starting point in the flyout. That seems smart, dynamic in nature, and reduces some clicks.

[ThomThomson]

Related to the funkiness of the new panels being appended to the bottom of the dashboard: #97064 & #108951

Both of these could potentially be fixed with a scroll to view call somewhere in the addEmbeddable function of Dashboard.

[ryankeairns]

It may be nice to have a secondary way of adding a panel where a user hovers/focus in the space between and sees something like this:

[nickpeihl]

One of the motivations for this proposal is actually to bring a little bit more of Lens into Dashboard to drive its adoption. We do this by directly presenting Lens' viz types to users. Also, we found that the word Lens itself (as included in the Select type dropdown) doesn't say much to new users.

One of the major pain points of the legacy visualizations is that users had to choose a viz type first. While building the viz you might discover the viz type isn't suitable for the data and you have to start over with a new viz type from scratch. Lens alleviates this by making it easier to switch viz types in the editor. And with Lens you shouldn't need to choose a viz type first. From the Lens documentation:

If you’re unsure about the visualization type you want to use, or how you want to display the data, drag the fields you want to visualize onto the workspace, then let Lens choose for you.

In this design the user on a dashboard chooses a viz type first that would open the Lens editor with the chosen viz type as the default. However, currently Lens will automatically change the viz type depending on which field(s) are added. So a user might have chosen a line chart, but Lens might change to a metric viz once the user adds a field. If the user enters Lens by choosing a viz type from the flyout, I think we have to tell the Lens editor "Hey, this user expected to create a line chart. Pin the viz type so that it doesn't change when the user adds a field. But also, allow the user to select a different viz type if they want to."

I'm undecided on whether the flyout should list the Lens viz types. On one hand, I like giving users the option to choose the viz type first. But having Lens suggest the viz type based on the field is also helpful. I think we're removing that viz type suggestion for any user entering Lens from the flyout.

[andreadelrio]

One of the motivations for this proposal is actually to bring a little bit more of Lens into Dashboard to drive its adoption. We do this by directly presenting Lens' viz types to users. Also, we found that the word Lens itself (as included in the Select type dropdown) doesn't say much to new users.

One of the major pain points of the legacy visualizations is that users had to choose a viz type first. While building the viz you might discover the viz type isn't suitable for the data and you have to start over with a new viz type from scratch. Lens alleviates this by making it easier to switch viz types in the editor. And with Lens you shouldn't need to choose a viz type first. From the Lens documentation:

If you’re unsure about the visualization type you want to use, or how you want to display the data, drag the fields you want to visualize onto the workspace, then let Lens choose for you.

In this design the user on a dashboard chooses a viz type first that would open the Lens editor with the chosen viz type as the default. However, currently Lens will automatically change the viz type depending on which field(s) are added. So a user might have chosen a line chart, but Lens might change to a metric viz once the user adds a field. If the user enters Lens by choosing a viz type from the flyout, I think we have to tell the Lens editor "Hey, this user expected to create a line chart. Pin the viz type so that it doesn't change when the user adds a field. But also, allow the user to select a different viz type if they want to."

I'm undecided on whether the flyout should list the Lens viz types. On one hand, I like giving users the option to choose the viz type first. But having Lens suggest the viz type based on the field is also helpful. I think we're removing that viz type suggestion for any user entering Lens from the flyout.

Interesting. I had no idea that was the behavior of Lens. I know it shows a "Suggestions" section below the visualization but I didn't know it actually changes the type of visualization for users at times. I'd be interested to see this behavior in action to understand it better and see how we can accommodate this proposal to it. I tried replicating it myself in Lens but I was unsuccessful.

[nickpeihl]

However, currently Lens will automatically change the viz type depending on which field(s) are added. So a user might have chosen a line chart, but Lens might change to a metric viz once the user adds a field.

@elastic/kibana-visualizations Is the above statement correct or am I misunderstanding?

[stratoula]

The Lens editor is trying to suggest the best chart depending on your fields. This might not be so obvious on the dataview mode because if I am not mistaken if you drop a field on the the default chart (stacked vertical bar), we are not suggesting something different (@dej611 correct me if I am wrong).
It is more obvious when you are on a text based mode (ESQL, SQL). For example here I am dropping a number field and it suggests the metric viz.

Another example from the dataview mode. I start with waffle but as I am dropping fields it changes on a treemap

With that being said I really like the idea to promote Lens and not the old editors, I am not sure though if it is a good idea to list all the charts that Lens supports. I don't have a super strong objection of course. I am open for discussion!

[cqliu1]

I prefer having the Lens vis types listed out to highlight the breadth of charts available. With all the types listed out, there are many more options/routes that lead to Lens and a higher probability that the user will end up in Lens when building a new visualization. My concern is just having a single "Lens" option would potentially lead to the same issue as we had in the 7.9 vis modal where Lens is somewhat hidden amongst many other options and loses prominence. Even if we're somewhat reverting to an older design that we moved away from, I think the issue then wasn't that we had chart types individually listed, but that most of the options listed were legacy vis types with only one Lens option. I also can't think of a good way to refer to Lens as a vis option without actually saying "Lens" because it may not be apparent that's the option you choose to build a new vis and may lead to confusion when you also have options like "Custom visualization" or "Aggregation based visualization", even if they're further down the list.

RE: Lens auto switching chart types from my observation, Lens will only switch to a different type when you add a field that is not compatible with the chart type you were on, and if the user wants to build a vis for that incompatible field, they probably wouldn't be upset that we didn't preserve their initial chart type selection. As long as we start with the selected chart type, I don't think it'll be too jarring when the chart type switches.

Side note: Not sure how many people run into the chart switching/how big of a pain point this is, but maybe Lens could give some context when switching chart types by displaying info toasts explaining why the chart changed?

[ThomThomson]

From the engineering side, we'll need to ensure that we tackle #97010 alongside this redesign. This will be especially important for serverless, because we will want to configure which menu items show up here, and we want to put that complexity in the plugins that register the embeddable type rather than in Dashboard.

[ThomThomson]

When we build this, we should be extra careful about focus management, as we have the opportunity to speed up dashboard usage by focusing the relevant area. #64864

[andreadelrio]

After conducting a dedicated survey to get user feedback on this idea, here are the designs we will be implementing. More details can be found in the Figma file.

Improved toolbar

Flyout

[andreadelrio]

For the first implementation of this, @teresaalvarezsoler can you confirm if you agree with these changes? and what you think about the following questions?

For the flyout

1. Rename flyout from Add to dashboard to Add panel
2. Remove the Filtering section from flyout
3. The flyout is triggered by the Add panel button in the toolbar
4. Do we still need a search box for the Add panel flyout? Given that we won't have a ton of options there, I'm thinking it shouldn't be a hard requirement
5. Can we remove the tabs (i.e. New | Library) and keep one flyout for Add panel and a separate one (which already exists) for Add from Library?
6. Do we still need the subheadings? (i.e. Visualizations, Annotations and Other)

For the toolbar

• The Controls button remains unchanged, both in its label and in the popover it triggers.

[teresaalvarezsoler]

Thank you @andreadelrio

Do we still need a search box for the Add panel flyout? Given that we won't have a ton of options there, I'm thinking it shouldn't be a hard requirement

I agree they seem too few options to have a flyout for serverless. I'm also worried that we will need to expose all the "Agg base" options for stateless in the flyout causing confusion to users. These options are currently hidden in the popover.

Let's then keep the popover in the Add panel button until we can remove TSVB and Agg based from stateless. @ThomThomson do you agree?

[ThomThomson]

@teresaalvarezsoler that works for me!

[cqliu1]

@teresaalvarezsoler To clarify, we're holding off on the flyout design until we can remove TSVB and Agg based vis, correct? For now, I will remove the quick actions and relabel the Select type button to Add panel.

[stratoula]

In serverless we have removed the agg based/TSVB editors and they don't appear on the dahsboard add panel popover.

[teresaalvarezsoler]

that's right @cqliu1 . Thank you! A couple of clarifications:

• The deprecated input controls won't appear anymore
• TSVB and Agg based will be the last options in the menu

[IanLee1521]

FWIW, if you're looking for user testing / feedback I would volunteer for this if you're needing someone.

I've actually been a bit confused with the current (8.12) difference between say "Create visualization" and "Add panel > Lens".

[ThomThomson]

Noting here @nreese's suggestion that as part of the PR that fixes this issue, we should also write a functional test that counts the number of entries in this menu and fails if one is added. This way, we can be pinged for review on any embeddable addition to the Dashboard.