Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/v0.2.0 #152

Merged
merged 40 commits into from
Feb 28, 2023
Merged

Feature/v0.2.0 #152

merged 40 commits into from
Feb 28, 2023

Conversation

racheldaniel
Copy link
Contributor

@racheldaniel racheldaniel commented Jan 9, 2023

What is this PR?

This is a:

  • documentation update
  • bug fix with no breaking changes
  • new functionality
  • a breaking change

All pull requests from community contributors should target the main branch (default).

Description & motivation

This branch will become server version 0.2.0 when merged to main. It is compatible with dbt-core versions >=1.5.0-prerelease. For use with other core versions, please use the 0.1.latest branch.

Docker images for use by the IDE and Exec API will only be built from this branch for core 1.5.0b1 and up, including HEAD once this is merged to main.

The primary reason for this breaking change is core's introduction of the new click API and plans to deprecate the lib.py module that server 0.1.0 uses as the entrypoint for dbt core. Our use of the core logger has also been deprecated, necessitating changes in this branch.

Changes in this branch:

  • All core command endpoints except for /compile are removed and consolidated under the async/dbt and /sync/dbt endpoints
  • Logger is updated to use new structured logging
  • Project path is an option for initial project parse in addition to the state_id construct
  • Logic added so that partial parsing works with use of a state_id (already would have worked with project path)
  • Updates FastAPI
  • Adds and updates tests

Checklist

  • I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
    • BigQuery
    • Postgres
    • Redshift
    • Snowflake
    • Databricks
    • Spark
  • I have updated the README.md (if applicable)
  • I have added tests & descriptions to my models
  • I have added an entry to CHANGELOG.md

@cla-bot cla-bot bot added the cla:yes label Jan 9, 2023
@github-actions
Copy link

github-actions bot commented Jan 9, 2023

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

ChenyuLInx and others added 19 commits January 12, 2023 10:59
* possible way of intergrating all of the dbt commands

* somewhat working version of a generalized framework

* working version of run, a lot of refactor and better core interface needed

* using some new interface

* remove unused function

* using state for run task

* some clean up

* Resolves merge conflicts (#145)

* Core integration updates (#148)

* Updates state_id usage

* Moves task logic to StateController

* removes hardcoded command

* Initiates logmanager in async function

* Removes old async logic and reinstates python logger for dbt-server

Co-authored-by: Rachel <[email protected]>
Co-authored-by: Rachel Daniel <[email protected]>
…irements.txt to resolve error handling issue with underlying FastAPI dependency (#149)

* Upgrade FastAPI version in requirements.txt and add httpx to dev-requirements.txt to resolve error handling issue with underlying FastAPI dependency

* Add changelog entry
* possible way of intergrating all of the dbt commands

* somewhat working version of a generalized framework

* working version of run, a lot of refactor and better core interface needed

* using some new interface

* remove unused function

* using state for run task

* some clean up

* Core integration updates (#148)

* Updates state_id usage

* Moves task logic to StateController

* removes hardcoded command

* Removes old async logic and reinstates python logger for dbt-server

* Beginning logic to accept a project path

* Adds project_path storing and cacheing

* Removes prints and fixes caching issue

* removes unused task functions

* adds changie entry

* removes dup code from rebase error

* removes dup code from rebase error

* removes dup code from rebase error

* Adds tests for dbt_entry and preliminary state tests

* Removes unused file

* Copies minimal project to tempdir to avoid writing files

Co-authored-by: Chenyu Li <[email protected]>
Co-authored-by: Chenyu Li <[email protected]>
* Updates async endpoint to use set_profile_name function

* Adds checkfirst flag to avoid table exists error

* Fixes profile name and potential fix for sqlalchemy error

* Adds profile back to command args

* Fixes whitespace

* Adds status endpoint
* Adds sync endpoint and fixes linting

* Adds test for sync dbt entry endpoint

* Fixes formatting

* Adds changie entry
* Add the requests library to the requirements

* Replace each specific task update method with a generic method so that it can be called cleanly upstream

* Update this class to use camel casing

* Add new update task status method that sets the task status in the local DB as well as calling the callback if there is one

* Accept a callback url and pass it to the async command method

* Call the new update task status method where the crud methods were previously called

* Move requests from the dev requirements to requirements

* Return the state ID in addition to the other task fields in the async response

* Remove commented out code

* Specify to retry post requests since it isn't enabled by default

* Update dbt_server/views.py

Co-authored-by: Rachel <[email protected]>

* Rename DBTCommandArgs to DbtCommandArgs

* Add a change log entry

---------

Co-authored-by: Rachel <[email protected]>
* Updates db path to working dir instead of app root

* Solidifies locations that the dbt-server writes to

* Changes back to app root after dbt command run

* Fixes comment
…creating the async task. If not present, create a task ID and use it (#168)
* Fix tests.

* Fix wrong package

* Remove adaptor requirements and skip tests without dependency.

* Fix wrong package name
@racheldaniel racheldaniel marked this pull request as ready for review February 24, 2023 16:23
racheldaniel and others added 13 commits February 27, 2023 09:23
* Resolves merge conflicts

* Cherry-pick gone awry

* spaces

* Allows images to build on PR
* Removes conditional on test, tailors to branch

* Adds changie entry

* Comments out unused matrix
* Add smoke test and check in test dbt project jaffle shop.

* nits

* nits

* Add changie

* Update smoke test.

* Start dbt-server inside smoke test.

* Fix format.
@racheldaniel racheldaniel merged commit c9190c0 into main Feb 28, 2023
@racheldaniel racheldaniel deleted the feature/v0.2.0 branch February 28, 2023 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants