[CT-2744] Refactor: entry point for cross-project ref
#7954
Labels
multi_project
tech_debt
Behind-the-scenes changes, with little direct impact on end-user functionality
After much spiking and iterating on how to handle "public models from an upstream project," we've landed on a really satisfying implementation (#7890): These are just models. We needn't add special-case logic (or duplicate boilerplate code) within
dbt-core
for handling these nodes: listing them, resolving references to them, (not) running them, etc.This implementation gives us a much narrower entry point within
dbt-core
for the resolution of cross-projectref
. In #6725 (reply in thread), I shared more context around our intent for this feature. It should be uniquely available for dbt Cloud (Enterprise), given the problem it's solving—collaboration across multiple teams, with dozens or hundreds of contributors. It is not a publicly committed interface within thedbt-core
framework. And it will yield a functional outcome comparable to that of using packages, via a mechanism that's significantly better at scale.In a separate discussion earlier this year (#7173), we shared our aspiration for a future in which
dbt-core
is a pluggable library: narrowly scoped to its core workflow, while also providing more flexible points for extensions. Now that the refactor above is complete—cleaner interfaces, more-precise entry point—we can experiment with putting all logic specific to cross-projectref
within a separate plugin.So, the scope of this issue is to finish consolidating the remnants of this logic within
dbt-core
; to provide ourselves with a single clean entry point; and to make a first attempt at implementing a plugin framework, on which we can keep iterating.The text was updated successfully, but these errors were encountered: