You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
The field renaming in #5775 means that a user upgrading from v1.2 → v1.3 will see a parsing error immediately after upgrading, if they have metrics already defined in their project:
$ dbt parse
dbt.exceptions.ParsingException: Parsing Error
Invalid metrics config given in FilePath(searched_path='models', relative_path='metric.yml', modification_time=1662969785.677178, project_root='/Users/jerco/dev/scratch/testy') @ metrics: {'name': 'new_customers', 'label': 'New Customers', 'model': "ref('dim_customers')", 'description': 'The number of paid customers using the product', 'type': 'count', 'sql': 'user_id', 'timestamp': 'signup_date', 'time_grains': ['day', 'week', 'month'], 'dimensions': ['plan', 'country'], 'filters': [{'field': 'is_paying', 'operator': 'is', 'value': 'true'}, {'field': 'lifetime_value', 'operator': '>=', 'value': '100'}, {'field': 'company_name', 'operator': '!=', 'value': "'Acme, Inc'"}, {'field': 'signup_date', 'operator': '>=', 'value': "'2020-01-01'"}], 'meta': {'team': 'Finance'}} - at path []: 'calculation_method' is a required property
This also breaks backwards compatibility for users of "Slim CI":
$ dbt ls -s state:modified --state state/
08:07:02 Encountered an error:
Field "metrics" of type Mapping[str, ParsedMetric] in WritableManifest has invalid value {'metric.my_project.new_customers': {'fqn': ['my_project', 'new_customers'], 'unique_id': 'metric.my_project.new_customers', 'package_name': 'my_project', 'root_path': '/Users/jerco/dev/scratch/testy', 'path': 'metric.yml', 'original_file_path': 'models/metric.yml', 'name': 'new_customers', 'description': 'The number of paid customers using the product', 'label': 'New Customers', 'type': 'count', 'sql': 'user_id', 'timestamp': 'signup_date', 'filters': [{'field': 'is_paying', 'operator': 'is', 'value': 'true'}, {'field': 'lifetime_value', 'operator': '>=', 'value': '100'}, {'field': 'company_name', 'operator': '!=', 'value': "'Acme, Inc'"}, {'field': 'signup_date', 'operator': '>=', 'value': "'2020-01-01'"}], 'time_grains': ['day', 'week', 'month'], 'dimensions': ['plan', 'country'], 'model': "ref('dim_customers')", 'model_unique_id': None, 'resource_type': 'metric', 'meta': {'team': 'Finance'}, 'tags': [], 'sources': [], 'depends_on': {'macros': [], 'nodes': ['model.my_project.dim_customers']}, 'refs': [['dim_customers']], 'metrics': [], 'created_at': 1662969788.598332}}
08:07:02 Traceback (most recent call last):
File "<string>", line 73, in from_dict
File "<string>", line 73, in <dictcomp>
File "<string>", line 76, in from_dict
mashumaro.exceptions.MissingField: Field "calculation_method" of type str is missing in ParsedMetric instance
Metrics are still an officially experimental feature, so we have reserved the right to do this — but we should aim for a smoother upgrade experience if possible.
Expected Behavior
Upgrading raises a deprecation warning, but does not immediately break. Specifically, if I have a metric definition containing attributes named sql and type, or type: expression, dbt should implicitly rename those and store those attributes with the new names/values instead.
Compatibility for --state comparison to older manifests, if possible. Could we extend the logic in upgrade_manifest_json to that effect? Otherwise, we should declare that v1.3 breaks forwards compatibility for manifests from v1.0-1.2.
What this might want to look like (quick & naive version): af4d10a
Testing:
Metrics with old attribute names, also type: expression → calculation_method: derived
We do have a functional test that checks for back/forward compatibility when using older manifests in Slim CI. We don't have a metric in that test project right now — perhaps we should! — would just require recreating the manifests for older versions (v4 through v7)
Is this a new bug in dbt-core?
Current Behavior
The field renaming in #5775 means that a user upgrading from v1.2 → v1.3 will see a parsing error immediately after upgrading, if they have
metrics
already defined in their project:This also breaks backwards compatibility for users of "Slim CI":
Metrics are still an officially experimental feature, so we have reserved the right to do this — but we should aim for a smoother upgrade experience if possible.
Expected Behavior
sql
andtype
, ortype: expression
, dbt should implicitly rename those and store those attributes with the new names/values instead.--state
comparison to older manifests, if possible. Could we extend the logic inupgrade_manifest_json
to that effect? Otherwise, we should declare that v1.3 breaks forwards compatibility for manifests from v1.0-1.2.Steps To Reproduce
dbt ls
), and save it somewhere locally (mv target/manifest.json state/
)dbt parse
dbt ls -s state:modified --state ...
(where...
is relative path to the previous manifest)Relevant log output
No response
Environment
Which database adapter are you using with dbt?
No response
Additional Context
No response
The text was updated successfully, but these errors were encountered: