Support partial update(similar to partial parsing) of a state via API?

[ChenyuLInx]

Create an endpoint that takes a state_id and content/information about a modifed file in the dbt-project(that the state_id refers to) to generate a new state in dbt-server.

This assumes that the content/information of the file is not too big to send through a request.

We will probably also need to figure out someway of making sure the files on IDE side and the generated state on server side are properly synced(for example: checksum of all file content on ide side and checksum of all content inside a state matches).

This is essentially mimicking the partial parsing that happens in CLI. And would be able to avoid sending the whole project over for ever file update.

[jtcohen6]

The parallel to partial parsing is apt. We already have logic for hashing + checking state, defined in build_manifest_state_check. (I also believe the comment on that method is out of date, and doesn't give us enough credit: dbt-labs/dbt-core#6536.)

The relevant inputs are:

• source file contents
• env vars
• vars (can be overridden via --vars — it's best to think of this similarly to source file contents, since it can dynamically "rewrite" source files, update project/profile config, change the shape of the DAG, ...)
• contents of active target (profile)
• dbt_project.yml (i.e. Project config, root + all packages)
• dbt-core version

Changes to some of those inputs would require a full re-parse. For the ones that support partial updates, we'd need to mimick the logic in dbt-core for which files need re-parsing. My sense is that the current approach to state_id with dbt-server does not take into account all of the same inputs.

[ChenyuLInx]

Close due to priority shift