[CT-1483] [Bug] Macros whose names contain "materialization" treated as materialization macros by compiler, fail with "no supported_languages..."

[CraftyIvan]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

After upgrading to dbt 1.3.0 which adds supported_languages for materialization macros (#5695), all standard macros whose name has "materlialization" as a substring prevent compilation.

A project I'm contributing to contains a macro named get_materialization (it defaults model materializations to view under certain conditions). The parser seems to treat it as a materialization macro for that, even though it's not (location of this in dbt-core code referenced below in "Additional Context").

So when running dbt run or dbt compile I get the following error:

No supported_languages found in materialization macro <project_prefix>__get_materialization

Expected Behavior

Only materialization macros are able to trigger this compilation error when not declaring supported languages.

A regular macro ({% macro_name([params]) %}) with the substring "materialization" in its name is not considered a materialization macro ({% materialization [materialization_name] ... %}) and does not trigger the compilation error.

Steps To Reproduce

1. Create a new regular macro and have its name contain the substring "materialization", e.g. get_materialization or print_materialization.
2. Run dbt compile.

Relevant log output

No response

Environment

- OS: macOS 12.5.1
- Python: 3.8.15
- dbt: 1.3.0

Which database adapter are you using with dbt?

snowflake

Additional Context

The parse_unparsed_macros method performs this "generalization":

dbt-core/core/dbt/parser/macros.py

Line 88 in f022435

if "materialization" in name:

[dbeatty10]

Thanks for surfacing this @CraftyIvan !

Agreed that this is a surprising result when there is a non-materialization macro that happens to have "materialization" within the name.

Our next step will be to determine what our options are for changing the implementation.

[dbeatty10]

Goal

• Only set the node.supported_languages configuration for materialization nodes
• Determine the node type independently of the name
  • so that non-materialization nodes may have "materialization" in their name without failing during compilation

Idea for resolution

@stu-k I saw you worked on this area of the code -- do you know if we could do something like the following? If not, do you have any other suggestions?

Replace:

dbt-core/core/dbt/parser/macros.py

Lines 87 to 90 in 9f280a8

	# get supported_languages for materialization macro
	if "materialization" in name:
	node.supported_languages = jinja.get_supported_languages(macro)
	yield node

with something like:

 # get supported_languages for materialization macro 
 if block.block_type_name == "materialization": 
     node.supported_languages = jinja.get_supported_languages(macro) 
 yield node 

[jtcohen6]

@dbeatty10 Your suggested change makes sense to me! Much better than the idea I was halfway through typing.

I'm going to tag this one for Team:Language, since that's whose surface area this falls under. Stu was original implementer here, so still very happy to hear from him too.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

[mjedrzejczak-wh]

please reopen this ticket, now we have version 1.5.2 and it is still not fixed :(

[jtcohen6]

Given how straightforward Doug's proposed fix is, I think this could be a community contribution.

@mjedrzejczak-wh Any interest in giving it a spin?

[mjedrzejczak-wh]

@jtcohen6 unfortunately we have in our projects such macros, which block us effectively from upgrade

and I would prefer not to fix this myself - I do not know dbt internals so well

[ArunT317]

@jtcohen6 @mjedrzejczak-wh @dbeatty10 @CraftyIvan @joevandyk

Just FYI, I too faced the same error while trying to upgrade dbt.

I did try the code change suggested by @dbeatty10 locally in my python virtual environment and it is working as expected. I am facing some other project specific errors but thought to share this here so that we are all aware that the idea proposed above is fixing the issue.

[fconrady]

I need to make materializations depend on a variable and this bug prevents me from giving the macro a correct name. Besides, the error is very confusing.

[mjedrzejczak-wh]

@fconrady it is already fixed and closed, update your dbt :)