Flytekit dbt plugin #1150
Merged
Flytekit dbt plugin #1150
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Supports dbt run and dbt test tasks * The plugin includes integration test that need local PostgreSQL database Signed-off-by: ariefrahmansyah <[email protected]>
Signed-off-by: ariefrahmansyah <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
This reverts commit 134fa1c. Signed-off-by: Eduardo Apolinario <[email protected]>
This reverts commit 02ef380. Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
eapolinario
requested review from
wild-endeavor,
kumare3 and
pingsutw
as code owners
Codecov Report
@@ Coverage Diff @@
## master #1150 +/- ##
==========================================
+ Coverage 68.35% 68.53% +0.18%
==========================================
Files 287 288 +1
Lines 25911 26197 +286
Branches 2900 2924 +24
==========================================
+ Hits 17711 17954 +243
- Misses 7721 7765 +44
+ Partials 479 478 -1
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Signed-off-by: Eduardo Apolinario <[email protected]>
| . | ||
| -e file:.#egg=flytekitplugins-dbt | ||
| # dbt-sqlite and dbt-core should be compatible | ||
| dbt-core==1.2.1 |
There was a problem hiding this comment.
can we not pin reqs in the .in file?
| pip install flytekitplugins-dbt | ||
| ``` | ||
|
|
||
| _Example coming soon!_ |
There was a problem hiding this comment.
how long do you think this will be here for? over under on a year?
There was a problem hiding this comment.
Good question. Do you think it's worth pointing this to a blog post or the flytesnacks examples? (when those are created obviously).
wild-endeavor
previously approved these changes
Sep 16, 2022
Signed-off-by: Eduardo Apolinario <[email protected]>
wild-endeavor
approved these changes
Sep 17, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TL;DR
This PR introduces a flytekit plugin for dbt
Type
Are all requirements met?
Complete description
This PR is heavily based on #886, with two exceptions: (1) we're using the
sqliteadapter to test the integration, and (2) minor changes to follow flytekit's style guide.It's worth noting that dbtlabs suggests we rely on parsing the output of the CLI as their python SDK is still very much in flux.
In terms of dbt functionality, this PR adds support for
dbt runanddbt test.You'll notice that we're using the jaffle shop example in the tests.
Tracking Issue
flyteorg/flyte#2202
Follow-up issue
NA