-
Notifications
You must be signed in to change notification settings - Fork 182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor common executor constructors with test coverage #774
Refactor common executor constructors with test coverage #774
Conversation
👷 Deploy Preview for amazing-pothos-a3bca0 processing.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #774 +/- ##
==========================================
+ Coverage 93.28% 94.62% +1.34%
==========================================
Files 55 55
Lines 2502 2419 -83
==========================================
- Hits 2334 2289 -45
+ Misses 168 130 -38 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a great improvement, both to tests and implementation. Thanks a lot for the refactor, @jbandoro !
I left a minor comment related to naming.
✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great improvement, thanks for the refactor, @jbandoro !
Bug fixes * Fix disable event tracking throwing error by @jbandoro in #784 * Fix support for string path for LoadMode.DBT_LS_FILE and docs by @Flinz in #788 * Remove stack trace to disable unnecessary K8s error by @tatiana in #790 Others * Update examples to use the astro-runtime 10.0.0 by @RNHTTR in #777 * Docs: add missing imports for mwaa getting started by @Benjamin0313 in #792 * Refactor common executor constructors with test coverage by @jbandoro in #774 * pre-commit updates in #789
**Bug fixes** * Fix disable event tracking throwing error by @jbandoro in #784 * Fix support for string path for `LoadMode.DBT_LS_FILE` and docs by @Flinz in #788 * Remove stack trace to disable unnecessary K8s error by @tatiana in #790 **Others** * Update examples to use the astro-runtime 10.0.0 by @RNHTTR in #777 * Docs: add missing imports for mwaa getting started by @Benjamin0313 in #792 * Refactor common executor constructors with test coverage by @jbandoro in #774 * pre-commit updates in #789
…ion base subclasses (#805) This fixes an issue reported in #804 after the refactor done in #774 where the `execute` methods for `DbtLocalBaseOperator`, `DbtDockerBaseOperator`, and `DbtKubernetesBaseOperator` were different. This PR refactors the `execute` method to the `AbstractDbtBaseOperator` so it's the same for all of the local, docker and kubernetes inherited operators, and adds `build_and_run_cmd` as an abstract method since the implementation is different across the 3 different execution modes. Closes #804
…ion base subclasses (#805) This fixes an issue reported in #804 after the refactor done in #774 where the `execute` methods for `DbtLocalBaseOperator`, `DbtDockerBaseOperator`, and `DbtKubernetesBaseOperator` were different. This PR refactors the `execute` method to the `AbstractDbtBaseOperator` so it's the same for all of the local, docker and kubernetes inherited operators, and adds `build_and_run_cmd` as an abstract method since the implementation is different across the 3 different execution modes. Closes #804 (cherry picked from commit 9c090a4)
) I noticed in astronomer#771 that there was a lot of repeated class constructors in order to add a new execution mode that is common among `local`, `docker` and `kubernetes` and there is no test coverage for the constructors and methods in some of the operators. This PR attempts to make it easier to add new execution operators in the future. ## Breaking Change? None There may be task UI color differences with the kuberentes/docker operators, since now all of LS/Seed/Run etc. operators across execution modes have the same task colors.
**Bug fixes** * Fix disable event tracking throwing error by @jbandoro in astronomer#784 * Fix support for string path for `LoadMode.DBT_LS_FILE` and docs by @Flinz in astronomer#788 * Remove stack trace to disable unnecessary K8s error by @tatiana in astronomer#790 **Others** * Update examples to use the astro-runtime 10.0.0 by @RNHTTR in astronomer#777 * Docs: add missing imports for mwaa getting started by @Benjamin0313 in astronomer#792 * Refactor common executor constructors with test coverage by @jbandoro in astronomer#774 * pre-commit updates in astronomer#789
…ion base subclasses (astronomer#805) This fixes an issue reported in astronomer#804 after the refactor done in astronomer#774 where the `execute` methods for `DbtLocalBaseOperator`, `DbtDockerBaseOperator`, and `DbtKubernetesBaseOperator` were different. This PR refactors the `execute` method to the `AbstractDbtBaseOperator` so it's the same for all of the local, docker and kubernetes inherited operators, and adds `build_and_run_cmd` as an abstract method since the implementation is different across the 3 different execution modes. Closes astronomer#804
Description
I noticed in #771 that there was a lot of repeated class constructors in order to add a new execution mode that is common among
local
,docker
andkubernetes
and there is no test coverage for the constructors and methods in some of the operators.This PR attempts to make it easier to add new execution operators in the future.
Related Issue(s)
None
Breaking Change?
None
There may be task UI color differences with the kuberentes/docker operators, since now all of LS/Seed/Run etc. operators across execution modes have the same task colors.
Checklist