[Explore] Adding custom expressions to adhoc metrics #4736

gabe-lyons · 2018-04-02T16:49:55Z

This is a followup to #4663. Sometimes you want more flexibility than the simple adhoc metric allows for. This PR adds a second tab that allows someone to define a metric as arbitrary sql. It also may be an easier UI for SQL lovers.

I also made sure the transitions between the two tabs are as seamless as possible, so from a user perspective it should feel like two views into the same metric definition.

demo of more advanced expressions:

demo of seamless transitions:

Unfortunately had to disable this feature on druid datasources because it seems like superset doesn't support querying druid via sql yet? Or does it? @betodealmeida @mistercrunch any idea how hard it would be to integrate this with druid?

test plan:

calculated some advanced metrics I could check (like SUM(value) + 100, SUM(value)/COUNT(DISTINCT target)) and verified that their outputs were correct as shown in video Add pandas-highcharts as requirements #1
checked transitions as shown in video Fix documentation #2
loaded druid datasource and verified the second tab did not appear
ran the specs

future work:

would love to integrate this with druid

reviewers:
@michellethomas @williaster @graceguo-supercat @mistercrunch

codecov-io · 2018-04-02T18:02:18Z

Codecov Report

Merging #4736 into master will increase coverage by 0.03%.
The diff coverage is 78.99%.

@@            Coverage Diff             @@
##           master    #4736      +/-   ##
==========================================
+ Coverage   72.33%   72.36%   +0.03%     
==========================================
  Files         208      208              
  Lines       15568    15662      +94     
  Branches     1204     1227      +23     
==========================================
+ Hits        11261    11334      +73     
- Misses       4304     4325      +21     
  Partials        3        3

Impacted Files	Coverage Δ
...ripts/explore/components/MetricDefinitionValue.jsx	`94.11% <ø> (+5.22%)`	⬆️
superset/utils.py	`88.1% <100%> (+0.02%)`	⬆️
superset/assets/javascripts/explore/constants.js	`100% <100%> (ø)`	⬆️
...s/javascripts/explore/propTypes/adhocMetricType.js	`100% <100%> (ø)`	⬆️
...pts/explore/components/controls/MetricsControl.jsx	`78.62% <100%> (-0.3%)`	⬇️
...javascripts/SqlLab/components/AceEditorWrapper.jsx	`76.05% <100%> (ø)`	⬆️
...vascripts/explore/components/AdhocMetricOption.jsx	`91.3% <66.66%> (-3.7%)`	⬇️
...ipts/explore/components/AdhocMetricEditPopover.jsx	`79.16% <78.43%> (-4.17%)`	⬇️
superset/connectors/sqla/models.py	`76.32% <9.09%> (-1.11%)`	⬇️
superset/assets/javascripts/explore/AdhocMetric.js	`94.91% <93.18%> (-5.09%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update daf9a3b...6a8dde6. Read the comment docs.

gabe-lyons · 2018-04-03T21:06:59Z

Updated based on comments from team at Airbnb-

a) now the smart defaults in the simple tab will only be inferred when the simple tab can entirely describe the advanced sql expression. if not, the simple tabs defaults will be blank.

b) the window can be resized if you want more working space in the advanced sql editor view

heres a gif describing both features:

williaster · 2018-04-03T21:19:14Z

@GabeLoins can we get a design pass on this before merging? I'm mostly concerned about the size of the tabs and the strange offset once you resize the popover container

gabe-lyons · 2018-04-03T23:18:45Z

@williaster good points. Adjusted tab widths and made it so the resize doesn't affect the arrow positioning. both features are demoed here:

mistercrunch · 2018-04-03T23:28:44Z

It's no biggy to me but tabs are usually used to organize heavy amounts of content, we probably could get away without using them at all. What about using the PopoverSection component?

gabe-lyons · 2018-04-04T00:53:03Z

@mistercrunch yeah I definitely considered that as well when working on it. The big motivator for going with tabs is that the metric definitions could easily hit 4-5 lines if someone was defining a case/if statment. I wanted to give the Ace editor a good amount of space by default so that would be possible.

williaster

a couple super tiny comments, looks great overall and a 🔥 feature!

williaster · 2018-04-11T21:00:20Z

superset/assets/javascripts/explore/AdhocMetric.js

+
+  isValid() {
+    if (this.expressionType === EXPRESSION_TYPES.SIMPLE) {
+      return !!(this.column && this.aggregate);


Boolean() also works fyi

Airbnb js style guide prefer !! to Boolean(...)- not sure if we have a different style guide for this project, but I like their advice on the whole.

https://github.com/airbnb/javascript#coercion--booleans

williaster · 2018-04-11T21:02:44Z

superset/assets/javascripts/explore/components/AdhocMetricEditPopover.jsx

-        aggregate: aggregate && aggregate.aggregate,
-      }),
-    });
+    this.setState({ adhocMetric: this.state.adhocMetric.duplicateWith({


nit -- I would newline adhocMetric, wish we had better lint rules enabled! 🙉

williaster · 2018-04-11T21:04:29Z

superset/assets/javascripts/explore/components/AdhocMetricEditPopover.jsx

+  }
+
+  componentWillUnmount() {
+    document.removeEventListener('mouseup', this.onMouseUp);


unlikely sitch, but could also call document.removeEventListener('mousemove', this.onMouseMove); to be safe

oooh good call

williaster · 2018-04-11T21:06:45Z

superset/assets/javascripts/explore/components/AdhocMetricEditPopover.jsx

-    const stateIsValid = this.state.adhocMetric.column && this.state.adhocMetric.aggregate;
-    const hasUnsavedChanges = this.state.adhocMetric.equals(this.props.adhocMetric);
+    const stateIsValid = adhocMetric.isValid();
+    const hasUnsavedChanges = adhocMetric.equals(propsAdhocMetric);


is this backwards? or I'm reading it wrong? I guess I'd expect equals to return true if they're the same which would mean no unsaved changes?

ah you're totally right- everything works because I'm also using it wrong in the render body 🤕

williaster · 2018-04-11T21:08:40Z

superset/assets/javascripts/explore/components/AdhocMetricOption.jsx

+  }
+
+  onPopoverResize() {
+   this.forceUpdate();


williaster · 2018-04-11T21:09:13Z

superset/assets/javascripts/explore/components/AdhocMetricOption.jsx

@@ -44,6 +50,7 @@ export default class AdhocMetricOption extends React.PureComponent {
        disabled
        overlay={overlay}
        rootClose
+        shouldUpdatePosition


🎉 thanks again for adding this!

williaster · 2018-04-11T21:11:11Z

superset/assets/spec/javascripts/explore/AdhocMetric_spec.js

@@ -82,4 +95,95 @@ describe('AdhocMetric', () => {

    expect(adhocMetric2.label).to.equal('label1');
  });
+
+  it('can determines if it is valid', () => {


nit remove determines 's'

michellethomas · 2018-04-12T21:26:05Z

superset/assets/javascripts/explore/AdhocMetric.js

+    this.expressionType = adhocMetric.expressionType || EXPRESSION_TYPES.SIMPLE;
+    if (this.expressionType === EXPRESSION_TYPES.SIMPLE) {
+      const inferredColumn = inferSqlExpressionColumn(adhocMetric);
+      this.column = adhocMetric.column || (inferredColumn && { column_name: inferredColumn });


Just to make sure I understand, inferSqlExpressionColumn is necessary if adhocMetric was a sql expression typed previously when EXPRESSION_TYPES was SQL correct? There's a lot going on in this section, maybe we could add a few comments to clarify?

gabe-lyons · 2018-04-13T00:32:33Z

@michellethomas @williaster addressed your comments, should be ready to merge!

williaster · 2018-04-13T18:20:43Z

thanks @GabeLoins! lgtm!

mistercrunch · 2018-04-13T21:04:43Z

* adding custom expressions to adhoc metrics * adjusted transitions and made the box expandable

gabe-lyons force-pushed the gabe_adding_custom_expressions_adhoc_metrics branch 3 times, most recently from 7643d21 to c0bf117 Compare April 2, 2018 17:40

adding custom expressions to adhoc metrics

1ac2d39

gabe-lyons force-pushed the gabe_adding_custom_expressions_adhoc_metrics branch from c0bf117 to fe895c5 Compare April 3, 2018 20:46

gabe-lyons force-pushed the gabe_adding_custom_expressions_adhoc_metrics branch from fe895c5 to 772e676 Compare April 3, 2018 23:12

williaster reviewed Apr 11, 2018

View reviewed changes

michellethomas reviewed Apr 12, 2018

View reviewed changes

gabe-lyons force-pushed the gabe_adding_custom_expressions_adhoc_metrics branch from 772e676 to db7b41f Compare April 13, 2018 00:31

gabe-lyons force-pushed the gabe_adding_custom_expressions_adhoc_metrics branch from db7b41f to 6a8dde6 Compare April 13, 2018 16:34

adjusted transitions and made the box expandable

6a8dde6

williaster merged commit 8669874 into apache:master Apr 13, 2018

gabe-lyons mentioned this pull request Apr 30, 2018

[Explore] Adding Adhoc Filters #4909

Merged

gabe-lyons mentioned this pull request May 21, 2018

Can't select the table's other metrics when explore the chart expect the count(*) metric #5045

Closed

3 tasks

michellethomas pushed a commit to michellethomas/panoramix that referenced this pull request May 24, 2018

[Explore] Adding custom expressions to adhoc metrics (apache#4736)

61fc716

* adding custom expressions to adhoc metrics * adjusted transitions and made the box expandable

timifasubaa pushed a commit to timifasubaa/incubator-superset that referenced this pull request May 31, 2018

[Explore] Adding custom expressions to adhoc metrics (apache#4736)

13fe8df

* adding custom expressions to adhoc metrics * adjusted transitions and made the box expandable

wenchma pushed a commit to wenchma/incubator-superset that referenced this pull request Nov 16, 2018

[Explore] Adding custom expressions to adhoc metrics (apache#4736)

0914cd6

* adding custom expressions to adhoc metrics * adjusted transitions and made the box expandable

sunnyKrystal mentioned this pull request Feb 27, 2019

Why we don't have other metrics but only the count(*) metric on metrics ? #5740

Closed

2 tasks

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.25.0 labels Feb 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Explore] Adding custom expressions to adhoc metrics #4736

[Explore] Adding custom expressions to adhoc metrics #4736

gabe-lyons commented Apr 2, 2018 •

edited

Loading

codecov-io commented Apr 2, 2018 •

edited

Loading

gabe-lyons commented Apr 3, 2018

williaster commented Apr 3, 2018 •

edited

Loading

gabe-lyons commented Apr 3, 2018

mistercrunch commented Apr 3, 2018

gabe-lyons commented Apr 4, 2018

williaster left a comment

williaster Apr 11, 2018

gabe-lyons Apr 11, 2018

williaster Apr 11, 2018

williaster Apr 11, 2018

gabe-lyons Apr 11, 2018

williaster Apr 11, 2018

gabe-lyons Apr 11, 2018

williaster Apr 11, 2018

williaster Apr 11, 2018

williaster Apr 11, 2018

michellethomas Apr 12, 2018

gabe-lyons commented Apr 13, 2018

williaster commented Apr 13, 2018

mistercrunch commented Apr 13, 2018

[Explore] Adding custom expressions to adhoc metrics #4736

[Explore] Adding custom expressions to adhoc metrics #4736

Conversation

gabe-lyons commented Apr 2, 2018 • edited Loading

codecov-io commented Apr 2, 2018 • edited Loading

Codecov Report

gabe-lyons commented Apr 3, 2018

williaster commented Apr 3, 2018 • edited Loading

gabe-lyons commented Apr 3, 2018

mistercrunch commented Apr 3, 2018

gabe-lyons commented Apr 4, 2018

williaster left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gabe-lyons commented Apr 13, 2018

williaster commented Apr 13, 2018

mistercrunch commented Apr 13, 2018

gabe-lyons commented Apr 2, 2018 •

edited

Loading

codecov-io commented Apr 2, 2018 •

edited

Loading

williaster commented Apr 3, 2018 •

edited

Loading