Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Config for dataset health check #12906

Merged

Conversation

graceguo-supercat
Copy link

SUMMARY

Found a couple issues when I tried to enable dataset health check in airbnb.

TEST PLAN

CI and manual test

ADDITIONAL INFORMATION

@@ -1100,7 +1104,3 @@ class CeleryConfig: # pylint: disable=too-few-public-methods
except Exception:
logger.exception("Found but failed to import local superset_config")
raise

# It's possible to add a dataset health check logic which is specific to your system.
Copy link
Author

@graceguo-supercat graceguo-supercat Feb 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config listed below the overwrite function won't be overwritten. Have to move it up.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment block here to stop others from adding new config below this line?

# -------------------------------------------------------------------
# *                WARNING:  STOP EDITING  HERE                    *
# -------------------------------------------------------------------
# Don't add config values below this line since local configs won't be
# able to override them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for that warning, we've seen issues with config values being added after overrides are loaded.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed!

@@ -704,7 +704,11 @@ def data(self) -> Dict[str, Any]:
data_["fetch_values_predicate"] = self.fetch_values_predicate
data_["template_params"] = self.template_params
data_["is_sqllab_view"] = self.is_sqllab_view
data_["health_check_message"] = self.health_check_message
data_["health_check_message"] = (
Copy link
Author

@graceguo-supercat graceguo-supercat Feb 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not return health check message, when feature is disabled but had populated health check message before.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add this comment to code as well?

# Don't return previously populated health check message in case
# the health check feature is turned off

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

Copy link
Member

@ktmud ktmud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some suggestions for code comments

@codecov-io
Copy link

Codecov Report

Merging #12906 (c2fefb6) into master (fab6238) will decrease coverage by 3.55%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #12906      +/-   ##
==========================================
- Coverage   66.97%   63.42%   -3.56%     
==========================================
  Files        1026      490     -536     
  Lines       50330    30256   -20074     
  Branches     5242        0    -5242     
==========================================
- Hits        33711    19189   -14522     
+ Misses      16485    11067    -5418     
+ Partials      134        0     -134     
Flag Coverage Δ
cypress ?
javascript ?
python 63.42% <100.00%> (-0.69%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/config.py 90.64% <100.00%> (ø)
superset/connectors/sqla/models.py 84.31% <100.00%> (-6.28%) ⬇️
superset/db_engines/hive.py 0.00% <0.00%> (-85.72%) ⬇️
superset/sql_validators/postgres.py 50.00% <0.00%> (-50.00%) ⬇️
superset/db_engine_specs/hive.py 73.84% <0.00%> (-17.31%) ⬇️
superset/databases/commands/create.py 83.67% <0.00%> (-8.17%) ⬇️
superset/databases/commands/update.py 85.71% <0.00%> (-8.17%) ⬇️
superset/db_engine_specs/sqlite.py 90.62% <0.00%> (-6.25%) ⬇️
superset/databases/commands/test_connection.py 84.78% <0.00%> (-4.35%) ⬇️
... and 550 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fab6238...c2fefb6. Read the comment docs.

@ktmud ktmud merged commit fd2d873 into apache:master Feb 3, 2021
amitmiran137 pushed a commit to nielsen-oss/superset that referenced this pull request Feb 3, 2021
* master: (23 commits)
  feat(explore): clear search on dataset change (apache#12909)
  chore: remove SIP-38 feature flag (apache#12894)
  fix: Config for dataset health check (apache#12906)
  fix(chart): allow null for most query object props (apache#12905)
  feat: add separate endpoint to fetch function names for autocomplete (apache#12840)
  chore: add required review on master (apache#12694)
  fix: comment typo (apache#12898)
  Migrates Radio component from Bootstrap to AntD. (apache#12738)
  fix: allow users to reset their passwords (apache#12886)
  fix(explore): missing select when groupby without metrics (apache#12890)
  refactor: dbapi exception mapping for dbapi's (apache#12869)
  feat(style-theme): add support for custom superset themes (apache#12858)
  chore(lint): fix pre-commit error (apache#12884)
  refactor(color-schemes): refactor setting of color schemes (apache#12857)
  feat(native-filters): Add defaultValue for Native filters modal (apache#12199)
  feat(release): add github token to changelog script (apache#12872)
  fix(menu): always show settings dropdown (apache#12877)
  Migrates Label component from Bootstrap to AntD. (apache#12774)
  [Helm] Automate datasource import (apache#10771)
  build: Skip loading example data from configs in CI (apache#12610)
  ...
serenajiang pushed a commit to airbnb/superset-fork that referenced this pull request Feb 3, 2021
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.2.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/S 🚢 1.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants