Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Druid lookup type extraction failing in pandas #3943

Closed
ghulands opened this issue Nov 27, 2017 · 1 comment
Closed

Druid lookup type extraction failing in pandas #3943

ghulands opened this issue Nov 27, 2017 · 1 comment

Comments

@ghulands
Copy link

ghulands commented Nov 27, 2017

When you have a druid lookup type column defined as

{
 "type": "extraction",
 "dimension": "coderizedColumn",
 "outputName": "prettyNameColumn",
 "outputType": "STRING",
 "extractionFn": {
   "type": "lookup",
   "dimension": "dimensionName",
   "outputName": "dimensionOutputName",
   "replaceMissingValueWith": "missing_value",
   "retainMissingValue": false,
   "lookup": {
     "type": "map",
     "map": {
       "Code1": "PrettyName1",
       "Code2": "PrettyName2",
     },
     "isOneToOne": false
   }
 }
}

When trying to query on it, superset now breaks with the following exception

2017-11-06 16:57:59,864:ERROR:root:"None of [[u'extractionFn', u'dimension', u'outputName', u'outputType', u'type']] are in the [index]"
Traceback (most recent call last):
 File "/app/.heroku/python/lib/python2.7/site-packages/superset/viz.py", line 275, in get_payload
   df = self.get_df()
 File "/app/.heroku/python/lib/python2.7/site-packages/superset/viz.py", line 98, in get_df
   self.results = self.datasource.query(query_obj)
 File "/app/.heroku/python/lib/python2.7/site-packages/superset/connectors/druid/models.py", line 1069, in query
   client=client, query_obj=query_obj, phase=2)
 File "/app/.heroku/python/lib/python2.7/site-packages/superset/connectors/druid/models.py", line 865, in get_query_str
   return self.run_query(client=client, phase=phase, **query_obj)
   qry['dimensions'], filters)
 File "/app/.heroku/python/lib/python2.7/site-packages/superset/connectors/druid/models.py", line 874, in _add_filter_from_pre_query_data
   f = Dimension(dim) == row[dim]
 File "/app/.heroku/python/lib/python2.7/site-packages/superset/connectors/druid/models.py", line 1006, in run_query
 File "/app/.heroku/python/lib/python2.7/site-packages/pandas/core/series.py", line 642, in __getitem__
   return self._get_with(key)
 File "/app/.heroku/python/lib/python2.7/site-packages/pandas/core/series.py", line 683, in _get_with
   return self.loc[key]
 File "/app/.heroku/python/lib/python2.7/site-packages/pandas/core/indexing.py", line 1328, in __getitem__
   return self._getitem_axis(key, axis=0)
 File "/app/.heroku/python/lib/python2.7/site-packages/pandas/core/indexing.py", line 1541, in _getitem_axis
   return self._getitem_iterable(key, axis=axis)
 File "/app/.heroku/python/lib/python2.7/site-packages/pandas/core/indexing.py", line 1081, in _getitem_iterable
   self._has_valid_type(key, axis)
 File "/app/.heroku/python/lib/python2.7/site-packages/pandas/core/indexing.py", line 1418, in _has_valid_type
   (key, self.obj._get_axis_name(axis)))
KeyError: "None of [[u'extractionFn', u'dimension', u'outputName', u'outputType', u'type']] are in the [index]"

This broke in the 0.19 series of releases.

@kkalyan
Copy link
Contributor

kkalyan commented Nov 28, 2017

This #3920 should fix issue with dimension spec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants