You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
(Here is where we set many of those attributes, during compilation)
The run_results.json produced by compile, docs generate, run/build/test/etc should include these fields (and only these fields) in addition to the node's unique_id. All other fields can be accessed from the manifest, and represent logical state at a point in time.
Background
Way back in v0.19, we removed the full node entry from run_results.json, and moved the compiled_code attribute into the manifest, so as to power dbt-docs from only manifest + catalog.
The manifest represents "logical" state (parsed from project code + configs)
Run results + catalog represent "applied" state (as materialized by dbt, as the objects exist in the data warehouse)
Compiled code can be a form of "applied" state. If a model's SQL depends on the results of an introspective query, it can vary given different inputs from the data warehouse.
Acceptance Criteria
The run_results.json produced by compile, docs generate, run, build, and test should include the compiled, compiled_code, relation_name, and depends_on.macros fields (and only these fields) in addition to the node's unique_id.
The same information should also be included in the results returned by dbtRunner when is used to invoke dbt as a library.
The text was updated successfully, but these errors were encountered:
github-actionsbot
changed the title
Include "compiled" node attributes in run results
[CT-2537] Include "compiled" node attributes in run results
May 5, 2023
@jtcohen6 per BLG: We're estimating this as written but have some concerns and would like to understand the use-case better-- Maybe a situation that would call for a new runtime artifact, compile_results.json?
Proposed change
Let's include attributes that can vary at compile/runtime. Those attributes are:
compiled: bool = True
(alwaysFalse
after parsing)compiled_code
(always empty after parsing) - this is most importantrelation_name
- this is set after parsing, but it can be updated for tests that have--store-failures
enableddepends_on.macros
([CT-500] Investigate differences in manifest.json output for 'compile' and 'run' commands #5079) - since additional macro dependencies are required/registered while a model is being materialized, versus just compiled (e.g.create_table_as
)(Here is where we set many of those attributes, during compilation)
The
run_results.json
produced bycompile
,docs generate
,run
/build
/test
/etc should include these fields (and only these fields) in addition to the node'sunique_id
. All other fields can be accessed from the manifest, and represent logical state at a point in time.Background
run_results.json
, and moved thecompiled_code
attribute into the manifest, so as to powerdbt-docs
from only manifest + catalog.Rationale
Compiled code can be a form of "applied" state. If a model's SQL depends on the results of an introspective query, it can vary given different inputs from the data warehouse.
Acceptance Criteria
run_results.json
produced bycompile
,docs generate
,run
,build
, andtest
should include thecompiled
,compiled_code
,relation_name
, anddepends_on.macros
fields (and only these fields) in addition to the node'sunique_id
.The text was updated successfully, but these errors were encountered: