You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Write a new 'publication artifact' as a result of dbt invocations. This artifact should contain the set of logical project state necessary for downstream projects to take dependencies (resolve refs) to public models in the producer project.
Minimally, the publication artifact would contain:
common artifact metadata (version, custom env vars, etc)
project name
quoting policy
the relation components (database, schema, alias) for each public model in the project
project-level dependencies (to prevent project-level cycles) or model-level dependencies (to prevent model-level cycles across projects)
Let's prove out that this publication artifact can be used to resolve two-variant refs, where the first variant is a project (not package) name. If the two-variant ref can be resolved with a package - prefer the package resolution to using the publication artifact.
The text was updated successfully, but these errors were encountered:
github-actionsbot
changed the title
[Spike] Produce publication artifact and resolve cross-project refs
[CT-2327] [Spike] Produce publication artifact and resolve cross-project refs
Mar 27, 2023
Paraphrasing from some conversations with @bossnunta
dbt list should include information about public models in upstream projects:
with a clear attribute saying that this is a node not in this project / an installed package (let's not call this external_nodes, since that's taken on a different meaning over the last ~year)
with the understanding that there will be significantly less information available about each of those nodes (only what's in the publication artifact), versus nodes defined in the project
Errors during cross-project ref
Will there be a way to tell from dbt-core when an error emerges from a bad "cross-project" reference, versus a bad intra-project reference?
When we're trying to resolve the ref, definitely!
If we can successfully resolve the ref, but then the user lacks permissions on the underlying database object... that would be trickier.
From the UX perspective, it could be compelling (P1) to tell users that their error comes from the fact that your current project does not have access to the referenced model in the upstream project. Then we can link you to a documentation page that tells you to:
if it's the same adapter type -- go set up permissions (Snowflake share, BigQuery project IAM, Redshift cross-db query, etc)
if it's a different adapter type -- we don't support this (yet)
Write a new 'publication artifact' as a result of dbt invocations. This artifact should contain the set of logical project state necessary for downstream projects to take dependencies (resolve refs) to public models in the producer project.
Minimally, the publication artifact would contain:
Let's prove out that this publication artifact can be used to resolve two-variant refs, where the first variant is a project (not package) name. If the two-variant ref can be resolved with a package - prefer the package resolution to using the publication artifact.
The text was updated successfully, but these errors were encountered: