feat(dbt): Support defining default values for columns and metrics #325
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new feature for the dbt sync: the ability to specify default configs for metrics and columns. This allows you to have default values at the model level, and then if needed overwrite these for specific columns and metrics.
To re-cap, there are 3 dbt sync behaviors:
Default (
reload_columns
is set toTrue
andmerge_metadata
is set toFalse
)All metrics defined in dbt are synced to Superset (if the metric already exists in Superset, it gets entirely replaced by dbt's metadata). Column definitions from dbt are also synced to Superset.
In this flow, metric default configs would be applied to all metrics (unless the config is defined at the metric level), and column default configs would be applied to all columns (unless the config is defined at the column level) - even columns that are not defined in dbt.
Merge metadata (
reload_columns
is set toFalse
andmerge_metadata
is set toTrue
)All metrics defined in dbt are synced to Superset (if the metric already exists in Superset, it gets entirely replaced by dbt's metadata). Column definitions from dbt are also synced to Superset.
In this flow, metric default configs would be applied to all metrics (even the ones that only exist in Superset), and column default configs would be applied to all columns (even columns not defined in dbt).
Preserve metadata (both
reload_columns and
merge_metadataare set to
False`)In this flow, Superset is considered the source of truth. Metrics are only synced from dbt if they don't exist in Superset yet. Column definitions are only synced from dbt if not defined in Superset yet.
Here, metric default configs are only applied to dbt metrics that are getting synced (new metrics), and to columns that are defined in dbt and getting synced.