-
Notifications
You must be signed in to change notification settings - Fork 967
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update 06-upgrading-to-v1.9.md #6535
Changes from 1 commit
130e00a
0535856
a93b497
8bf10fc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -29,7 +29,8 @@ Features and functionality new in dbt v1.9. | |||||
### Microbatch `incremental_strategy` | ||||||
|
||||||
:::info | ||||||
While microbatch is in "beta", this functionality is still gated behind an env var, which will change to a behavior flag when 1.9 is GA. To use microbatch, set `DBT_EXPERIMENTAL_MICROBATCH` to `true` wherever you're running dbt Core. | ||||||
|
||||||
If you use a custom microbatch macro, set the [`require_batched_execution_for_custom_microbatch_strategy`](/reference/global-configs/behavior-changes#custom-microbatch-strategy) behavior flag in your `dbt_project.yml` to enable batched execution. If you don't have a custom microbatch macro, you don't need to set this flag as dbt will handle microbatching automatically for any model using the microbatch strategy. | ||||||
::: | ||||||
|
||||||
Incremental models are, and have always been, a *performance optimization* — for datasets that are too large to be dropped and recreated from scratch every time you do a `dbt run`. Learn more about [incremental models](/docs/build/incremental-models-overview). | ||||||
|
@@ -83,6 +84,7 @@ You can read more about each of these behavior changes in the following links: | |||||
- (Introduced, disabled by default) [`skip_nodes_if_on_run_start_fails` project config flag](/reference/global-configs/behavior-changes#behavior-change-flags). If the flag is set and **any** `on-run-start` hook fails, mark all selected nodes as skipped. | ||||||
- `on-run-start/end` hooks are **always** run, regardless of whether they passed or failed last time. | ||||||
- (Introduced, disabled by default) [[Redshift] `restrict_direct_pg_catalog_access`](/reference/global-configs/behavior-changes#redshift-restrict_direct_pg_catalog_access). If the flag is set the adapter will use the Redshift API (through the Python client) if available, or query Redshift's `information_schema` tables instead of using `pg_` tables. | ||||||
- (Introduced, disabled by default) [`require_batched_execution_for_custom_microbatch_strategy`](/reference/global-configs/behavior-changes#custom-microbatch-strategy). Set to `True` in your `dbt_project.yml` if you use a custom microbatch macro to enable batched execution. If you don't have a custom microbatch macro, you don't need to set this flag as dbt will handle microbatching automatically for any model using the microbatch strategy. | ||||||
mirnawong1 marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we either link to the section that explains what There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it looks like we've been using this language in previous upgrade guides so happy to create an issue for someone on the team to address so we don't block this pr: https://docs.getdbt.com/docs/dbt-versions/core-upgrade/upgrading-to-v1.8#managing-changes-to-legacy-behaviors There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. created an issue here: #6543 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In addition to Grace's feedback we can potentially shorten this since we already stated above
Suggested change
|
||||||
|
||||||
## Adapter specific features and functionalities | ||||||
|
||||||
|
@@ -92,7 +94,7 @@ You can read more about each of these behavior changes in the following links: | |||||
|
||||||
### Snowflake | ||||||
|
||||||
- Iceberg Table Format support will be available on three out of the box materializations: table, incremental, dynamic tables. | ||||||
- Iceberg Table Format support will be available on three out-of-the-box materializations: table, incremental, dynamic tables. | ||||||
|
||||||
### Bigquery | ||||||
|
||||||
|
@@ -107,7 +109,7 @@ You can read more about each of these behavior changes in the following links: | |||||
|
||||||
We also made some quality-of-life improvements in Core 1.9, enabling you to: | ||||||
|
||||||
- Maintain data quality now that dbt returns an an error (versioned models) or warning (unversioned models) when someone [removes a contracted model by deleting, renaming, or disabling](/docs/collaborate/govern/model-contracts#how-are-breaking-changes-handled) it. | ||||||
- Maintain data quality now that dbt returns an error (versioned models) or warning (unversioned models) when someone [removes a contracted model by deleting, renaming, or disabling](/docs/collaborate/govern/model-contracts#how-are-breaking-changes-handled) it. | ||||||
- Document [data tests](/reference/resource-properties/description). | ||||||
- Use `ref` and `source` in [foreign key constraints](/reference/resource-properties/constraints). | ||||||
- Use `dbt test` with the `--resource-type` / `--exclude-resource-type` flag, making it possible to include or exclude data tests (`test`) or unit tests (`unit_test`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this just be regular text?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it could! but i also think it would benefit from having a callout so it's not as buried considering people do have custom macros and we don't want them to miss this.