[Epic] Applied State (part 1)

[peterallenwebb]

This epic comprises the changes to dbt-core necessary to support better real-time visibility into the current state of the database.

Real-Time State Updates

Beta Give feedback

1. [CT-2942] [Applied State] State-Update Events #8325
   +0
2. [CT-1905] [Spike] Get model "catalog" info after building, and fire in an event #6732
   Impact: CA Impact: Orch Refinement Team:Adapters logging performance +6
3. [CT-2537] Include "compiled" node attributes in run results #7519
   artifacts enhancement +2
   

The Performance Analysis and Monitoring portion of this epic will establish whether our current state tracking code, particularly dbt docs generate, is contributing to excess memory consumption. It will also us to quantify performance improvements across the Applied State project and guard against future performance regressions.

Performance Analysis and Monitoring

Beta Give feedback

1. [CT-2950] [Applied State] Fire ResourceReport Event at End of Run #8342
   +0
   
2. [CT-2941] [Applied State] Develop a Performance Analysis GitHub Action #8323
   +0
   
3. Review of Production OOMs w/ Categorization and/or Recommendations for Instrumentation

Performance Tuning

Beta Give feedback

1. [CT-458] Catalog queries should filter on specific relations in busy schemas #4997
   Impact: Adapters Team:Adapters enhancement performance +4
   
2. [CT-3050] [Applied State] Implement Core Changes for Filtered Catalog Queries #8521
   Impact: Adapters performance user docs +3
   
3. [CT-1303] Respect node selection in catalog queries run by docs generate #6014
   Team:Adapters enhancement node selection performance user docs +5
   
4. [CT-3118] Design Core Changes for Metadata-Driven Freshness #8636
   user docs +1
   
5. [CT-3149] [Implementation] allow source freshness to be evaluated from warehouse metadata tables, instead of running select max(loaded_at_field) ... SQL queries #8704
   Impact: Adapters enhancement user docs +3
   
6. [Bug] catalog.json includes all nodes even when no nodes are selected during dbt docs generate #8794
   backport 1.7.latest bug +2
   
7. [CT-2144] [Feature] Allow source freshness to be evaluated from table metadata when adapter supports it #7012
   Impact: Adapters enhancement +2
   

[graciegoheen]

Closing this to open a part 2 of this epic for 1.8