[CT-2374] [Bug] global materialization attribute overrides more specific package attributes

[sfc-gh-afedorov]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

brooklyn-data/dbt_artifacts#284

when I define

models:
  +materialized: view

this overrides a package's more specific

models:
  dbt_artifacts:
    sources:
      +materalized: incremental

which seems like something that violates some general compartmentalization principle

Expected Behavior

more specific attribute definitions should override more general globally defined defaults

Steps To Reproduce

1. create minimal dbt setup with dbt_artifacts installed
2. update yaml files as above
3. invoke dbt run --select dbt_artifacts

Relevant log output

first -

03:13:44  3 of 34 START sql view model afedorov.model_executions ......................... [RUN]

and then -

03:13:50  on-run-end failed, error:
03:13:50   001759 (42601): SQL compilation error:
03:13:50  INSERT statement's target must be a table

Environment

- OS: macos
- Python: 3.9.12
- dbt: 1.4.4

Which database adapter are you using with dbt?

snowflake

Additional Context

[jtcohen6]

It's nestled pretty deeply in the docs, but it is in there:

Configurations made in your dbt_project.yml file will override any configurations in a package (either in the dbt_project.yml file of the package, or in config blocks).

https://docs.getdbt.com/docs/build/packages#configuring-packages

[dbeatty10]

@sfc-gh-afedorov Thanks for opening this !

As @jtcohen6 mentioned, you are observing behavior that we expect, and I'll try to explain why.

Order of precedence

The order of precedence for configs is as follows (highest priority first):

1. Configuration within your dbt project:
   1. config() Jinja macro within a model
   2. config resource property in a .yml file
   3. dbt_project.yml file, under the models: key -- deepest nested being highest priority
2. Configuration within an installed package ("second verse, same as the first"):
   1. config() Jinja macro within a model
   2. config resource property in a .yml file
   3. dbt_project.yml file, under the models: key -- deepest nested being highest priority

Reason

One purpose of the dbt_project.yml file is specifically to allow the override of configurations of installed packages. Rather than forcing you to apply these overrides at the lowest level of granularity, it allows you to specify those overrides at the highest level that meets your needs.

Example

You can quickly and easily disable all models from the dbt_artifacts package at a high level of granularity within your dbt_packages.yml like this:

models:
  dbt_artifacts:
    +enabled: false

If the config across your dbt project and all the installed packages were fully flattened, then you would need to apply any overrides at the most granular level possible (model-by-model) in order for the overrides to take effect.

Summary

I'm going to close this as "not planned" since this is the intended behavior rather than a bug.

I've personally ran into this same thing before without realizing it, and I opened the following issue in a separate repo to warn when this situation arises:

• Warn about top-level models configs in dbt_project.yml dbt-project-evaluator#323

Also opened this issue to improve the docs:

• Global materialized config overrides more specific package config docs.getdbt.com#3148

[sfc-gh-afedorov]

Got it, thanks for the thorough explanation and thinking this through. A warning would have been a good alternative in my case, as well.