feat(visualization): add timeRangeFixed to Big Number w/ Trendlines

[ktmud]

CATEGORY

• Enhancement (new features, refinement)

SUMMARY

The Big Number chart has two problems related to the handling of null values:

1. Misleading time range: When there is no data at the beginning or ending periods of the filtered time range, the backend returns no data records for those periods, therefore the trend line will only start rendering from the datetime where there is data. This often leads to confusion or misinterpretations, especially then in the dashboards that have multiple trend line charts filtered to the same time range. They could all be showing similar smooth lines, yet representing very different time ranges.
2. NULL value at the last record: When the last record returns NULL value for the metric, the Big Number text will render as "No data", but the trend line still renders. Some users have requested to fallback displaying the Big Number with the last available dates.

This PR adds option timeRangeFixed to fix the first problem and adds a "fallback to last available value" behavior to address the second problem.

The timeRangedFixed option is disabled by default. When enabled, the empty starting and ending periods will take space on the x-axis, but there will be no connected lines, dots, or tooltips, just as if the metric values are NULL.

In case the last record has a NULL value for the metric, we will go back to the last available value and display it in grayed out style (see screen shots below).

The Flask API (viz.py is updated to return query.from_dttm and query.to_dttm for identifying time ranges.

The PR also updates how dates in tooltips are rendered. It will now render tooltips more rigidly by using the specified Time Grain, instead of guessing the granularity from the timestamp itself, which often fails to deal with quarters, e.g. a quarterly floored timestamp 2010-01-01 00:00:00 would be rendered as "2010", and 2010-04-01 00:00:00 rendered as "Apr 2010". They will now be rendered as "2010 Q1" and "2010 Q2" respectively. Note for Time Grain = month, "Apr 2010" will also be rendered as "2010-04" to be more i18n friendly. We should prefer more universal time formatting until we find a way to set locales for d3-format.

Will have to wait for apache-superset/superset-ui-plugins#403 to be merged and published.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before

This is a query against this data source:

SELECT TIMESTAMP '2019-01-01' as date, NULL as value
UNION
SELECT TIMESTAMP '2019-02-01' as date, 5 as value
UNION
SELECT TIMESTAMP '2019-03-01' as date, NULL  as value
UNION
SELECT TIMESTAMP '2019-04-01' as date, 3  as value
UNION
SELECT TIMESTAMP '2019-05-01' as date, 9  as value
UNION
SELECT TIMESTAMP '2019-06-01' as date, NULL as value

1. If the latest record is NULL, it will not fallback to last available datetime.
2. Even though the Time filter specified "Last year", it only shows a couple of months (till June 2019) we had data (even though June 2019 is null).

This query is incorporated in the dashboard below as "Sample Series":

After

It will now fallback to the last available value and show a warning:

The default font size is also increased to make the numbers even more noticeable (this has no effect on existing charts).

For the behavior about timeRangeFixed, check apache-superset/superset-ui-plugins#403

TEST PLAN

1. Save above SQL query in SQL Lab
2. Explore the data and change visualization type to "Big Number with Trendline"
3. Play with the date ranges and the new options and make sure everything works as expected.

ADDITIONAL INFORMATION

• Has associated issue: feat(big-number): allow fallback to last available value and fix time range for trend lines apache-superset/superset-ui-plugins#403
• Changes UI
• Requires DB Migration.
• Confirm DB Migration upgrade and downgrade tested.
• Introduces new feature or API
• Removes existing feature or API

REVIEWERS

@kristw @etr2460

[ktmud]

I'm adding a visibility API for control configs to conditionally hide certain controls. This makes sense for options that interact with each other.

[kristw]

Will get to this after apache-superset/superset-ui-plugins#403 is merged

[kristw]

remove comment

[ktmud]

This is a global change. It changes the default time range on the landing from the "Explore" chart button in SQL Lab from "100 years ago" to "No filter". It will affect all visualizations.

[kristw]

Oh. Do you want to save this as a separate PR?

[kristw]

It is small code change but has separated effect.

[ktmud]

It is somewhat related though. With the default "100 years ago" filter, applying the fixed range will not look good on the trend line. This change was motivated by the changes to Big Number.

I can create a separate PR nonetheless.

[codecov-io]

Codecov Report

Merging #9341 into master will decrease coverage by 0.01%.
The diff coverage is 68.18%.

@@            Coverage Diff             @@
##           master    #9341      +/-   ##
==========================================
- Coverage   59.03%   59.02%   -0.02%     
==========================================
  Files         379      379              
  Lines       12181    12189       +8     
  Branches     3010     3014       +4     
==========================================
+ Hits         7191     7194       +3     
- Misses       4809     4814       +5     
  Partials      181      181              

Impacted Files	Coverage Δ
superset-frontend/src/explore/controls.jsx	38.80% <0.00%> (-0.89%)	⬇️
.../src/explore/components/ControlPanelsContainer.jsx	66.66% <66.66%> (ø)
superset-frontend/src/explore/controlUtils.js	89.36% <81.25%> (-0.86%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 893c955...783efa4. Read the comment docs.