Confusion over metric definitions erodes trust

[marcushyett-ph]

Is your feature request related to a problem?

We receive a significant number of issues that relate to someone feeling the data is incorrect but actually the data is fine rather the data is being aggregated in a way they were not expecting - so the number is different to they expected.

Example 1: Moving from trends to funnels

By default we show a total count of metrics in trends, e.g. total number of page views in a day.

However if you switch to using funnels we also switch the aggregation to unique users (this is standard behavior for a funnel product). This shows a wildly different number, e.g. 100,000 page views a day, might only come from 10,000 users - and clicking between them makes it look like you've just lost 90,000 events and our data is unreliable.

In a user's own words:
"In the trends section...we have (a large number) of events...However, when trying to create a funnel with the exact same parameters...the specific event only shows (a small number) occurrences. I have debugged this for hours...and I cannot find anything that would create this behavior"

Example 2: Persons modal vs trends
Similar to example 1, if I look at a trends graph I may see the total number of events at any point in time (depending on how I configure it). However as soon as I click on a data point I get a modal with a list of the people who completed this event. This is going to be much smaller than the number of events which occurred (and can make it feel like data is missing).

Describe the solution you'd like

I'd love a consistent solution that makes it abundantly clear how any metric is being processed or aggregated to avoid confusion when using a different aggregation of the same metric.

Hacky quick fixes I have in mind

1. Where we use unique users by default - we also show the Total Count

In the persons modal

When I open a funnel (N.B. this is super-ugly there should be a way to do this much better)

2. We visually signify the type of aggregation in use in a way its unmissable

(again very ugly version - there must be a nicer way of doing this for funnels)

Describe alternatives you've considered

@clarkus has already briefly shared a few options to consider over DM - would love to explore these futher

Additional details

Would appreciate any feedback on the quick fixes proposed and more robust solutions

[clarkus]

related #6254

[clarkus]

I think the major attribute to clarify in these cases is the unit / aggregation method being applied to the graph series / funnel step / etc. This could be as simple as labeling values unique pageviews instead of pageviews. This is further complicated if you look at the related issue #6254 when secondary modifiers are applied to graph series - specifically formula can combine one or more graph series into a compound graph series item. The work in #5229 approaches a solution for this, but there does appear to be a need for a more fundamental change.

The solutions @marcushyett-ph is describing are great at adding context to the metrics shown. I think that is a solution worth pursuing. Another option would be adding descriptions to the aggregation methods in the selection lists. This would provide context directly when composing the query. Fundamentally, I think we need some component that can represent the disparate means of defining a graph series item. I'll work on that as well as some simple descriptions for aggregation methods.