Fix: ensure operator execute method is consistent across all execution base subclasses
#805
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.
|
dosubot
bot
added
size:M
jbandoro
changed the title
execute method is the same across all operator typesexecute method is consistent across all execution base subclasses
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #805 +/- ##
==========================================
+ Coverage 94.69% 94.76% +0.07%
==========================================
Files 55 55
Lines 2449 2447 -2
==========================================
Hits 2319 2319
+ Misses 130 128 -2 ☔ View full report in Codecov by Sentry. |
This reverts commit caae7cb.
tatiana
pushed a commit
that referenced
this pull request
Jan 26, 2024
…ion base subclasses (#805) This fixes an issue reported in #804 after the refactor done in #774 where the `execute` methods for `DbtLocalBaseOperator`, `DbtDockerBaseOperator`, and `DbtKubernetesBaseOperator` were different. This PR refactors the `execute` method to the `AbstractDbtBaseOperator` so it's the same for all of the local, docker and kubernetes inherited operators, and adds `build_and_run_cmd` as an abstract method since the implementation is different across the 3 different execution modes. Closes #804 (cherry picked from commit 9c090a4)
tatiana
added a commit
that referenced
this pull request
Jan 26, 2024
Bug fixes * Fix: ensure DbtGraph.update_node_dependency is called for all load methods by @jbandoro in #803 * Fix: ensure operator execute method is consistent across all execution base subclasses by @jbandoro in #805 * Fix custom selector when test node has no depends_on values by @tatiana in #814 * Fix forwarding selectors to test task when using TestBehavior.AFTER_ALL (#816) Others * Docs: Remove incorrect docstring from DbtLocalBaseOperator by @jakob-hvitnov-telia in #797 * Add more logs to troubleshoot custom selector by @tatiana in #809 * Fix OpenLineage integration documentation by @tatiana in #810 * Fix test dependencies after Airflow 2.8 release by @jbandoro and @tatiana in #806 * Use Airflow constraint file for test environment setup by @jbandoro in #812 * pre-commit updates in #799, #807
tatiana
added a commit
that referenced
this pull request
Jan 26, 2024
Bug fixes * Fix: ensure DbtGraph.update_node_dependency is called for all load methods by @jbandoro in #803 * Fix: ensure operator execute method is consistent across all execution base subclasses by @jbandoro in #805 * Fix custom selector when test node has no depends_on values by @tatiana in #814 * Fix forwarding selectors to test task when using TestBehavior.AFTER_ALL (#816) Others * Docs: Remove incorrect docstring from DbtLocalBaseOperator by @jakob-hvitnov-telia in #797 * Add more logs to troubleshoot custom selector by @tatiana in #809 * Fix OpenLineage integration documentation by @tatiana in #810 * Fix test dependencies after Airflow 2.8 release by @jbandoro and @tatiana in #806 * Use Airflow constraint file for test environment setup by @jbandoro in #812 * pre-commit updates in #799, #807
tatiana
added a commit
that referenced
this pull request
Jan 26, 2024
Bug fixes * Fix: ensure DbtGraph.update_node_dependency is called for all load methods by @jbandoro in #803 * Fix: ensure operator execute method is consistent across all execution base subclasses by @jbandoro in #805 * Fix custom selector when test node has no depends_on values by @tatiana in #814 * Fix forwarding selectors to test task when using TestBehavior.AFTER_ALL by @tatiana in #816 Others * Docs: Remove incorrect docstring from DbtLocalBaseOperator by @jakob-hvitnov-telia in #797 * Add more logs to troubleshoot custom selector by @tatiana in #809 * Fix OpenLineage integration documentation by @tatiana in #810 * Fix test dependencies after Airflow 2.8 release by @jbandoro and @tatiana in #806 * Use Airflow constraint file for test environment setup by @jbandoro in #812 * pre-commit updates in #799, #807
Merged
tatiana
added a commit
that referenced
this pull request
Jan 26, 2024
**Bug fixes** * Fix: ensure ``DbtGraph.update_node_dependency`` is called for all load methods by @jbandoro in #803 * Fix: ensure operator ``execute`` method is consistent across all execution base subclasses by @jbandoro in #805 * Fix custom selector when ``test`` node has no ``depends_on`` values by @tatiana in #814 * Fix forwarding selectors to test task when using ``TestBehavior.AFTER_ALL`` by @tatiana in #816 **Others** * Docs: Remove incorrect docstring from ``DbtLocalBaseOperator`` by @jakob-hvitnov-telia in #797 * Add more logs to troubleshoot custom selector by @tatiana in #809 * Fix OpenLineage integration documentation by @tatiana in #810 * Fix test dependencies after Airflow 2.8 release by @jbandoro and @tatiana in #806 * Use Airflow constraint file for test environment setup by @jbandoro in #812 * pre-commit updates in #799, #807
tatiana
added a commit
that referenced
this pull request
Jan 26, 2024
Bug fixes * Fix: ensure DbtGraph.update_node_dependency is called for all load methods by @jbandoro in #803 * Fix: ensure operator execute method is consistent across all execution base subclasses by @jbandoro in #805 * Fix custom selector when test node has no depends_on values by @tatiana in #814 * Fix forwarding selectors to test task when using TestBehavior.AFTER_ALL by @tatiana in #816 Others * Docs: Remove incorrect docstring from DbtLocalBaseOperator by @jakob-hvitnov-telia in #797 * Add more logs to troubleshoot custom selector by @tatiana in #809 * Fix OpenLineage integration documentation by @tatiana in #810 * Fix test dependencies after Airflow 2.8 release by @jbandoro and @tatiana in #806 * Use Airflow constraint file for test environment setup by @jbandoro in #812 * pre-commit updates in #799, #807
Merged
tatiana
added a commit
that referenced
this pull request
Jan 27, 2024
**Bug fixes** * Fix: ensure ``DbtGraph.update_node_dependency`` is called for all load methods by @jbandoro in #803 * Fix: ensure operator ``execute`` method is consistent across all execution base subclasses by @jbandoro in #805 * Fix custom selector when ``test`` node has no ``depends_on`` values by @tatiana in #814 * Fix forwarding selectors to test task when using ``TestBehavior.AFTER_ALL`` by @tatiana in #816 **Others** * Docs: Remove incorrect docstring from ``DbtLocalBaseOperator`` by @jakob-hvitnov-telia in #797 * Add more logs to troubleshoot custom selector by @tatiana in #809 * Fix OpenLineage integration documentation by @tatiana in #810 * Fix test dependencies after Airflow 2.8 release by @jbandoro and @tatiana in #806 * Use Airflow constraint file for test environment setup by @jbandoro in #812 * pre-commit updates in #799, #807 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Justin Bandoro <[email protected]> Co-authored-by: Jakob Aron Hvitnov <[email protected]>
arojasb3
pushed a commit
to arojasb3/astronomer-cosmos
that referenced
this pull request
Jul 14, 2024
…ion base subclasses (astronomer#805) This fixes an issue reported in astronomer#804 after the refactor done in astronomer#774 where the `execute` methods for `DbtLocalBaseOperator`, `DbtDockerBaseOperator`, and `DbtKubernetesBaseOperator` were different. This PR refactors the `execute` method to the `AbstractDbtBaseOperator` so it's the same for all of the local, docker and kubernetes inherited operators, and adds `build_and_run_cmd` as an abstract method since the implementation is different across the 3 different execution modes. Closes astronomer#804
arojasb3
pushed a commit
to arojasb3/astronomer-cosmos
that referenced
this pull request
Jul 14, 2024
**Bug fixes** * Fix: ensure ``DbtGraph.update_node_dependency`` is called for all load methods by @jbandoro in astronomer#803 * Fix: ensure operator ``execute`` method is consistent across all execution base subclasses by @jbandoro in astronomer#805 * Fix custom selector when ``test`` node has no ``depends_on`` values by @tatiana in astronomer#814 * Fix forwarding selectors to test task when using ``TestBehavior.AFTER_ALL`` by @tatiana in astronomer#816 **Others** * Docs: Remove incorrect docstring from ``DbtLocalBaseOperator`` by @jakob-hvitnov-telia in astronomer#797 * Add more logs to troubleshoot custom selector by @tatiana in astronomer#809 * Fix OpenLineage integration documentation by @tatiana in astronomer#810 * Fix test dependencies after Airflow 2.8 release by @jbandoro and @tatiana in astronomer#806 * Use Airflow constraint file for test environment setup by @jbandoro in astronomer#812 * pre-commit updates in astronomer#799, astronomer#807 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Justin Bandoro <[email protected]> Co-authored-by: Jakob Aron Hvitnov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This fixes an issue reported in #804 after the refactor done in #774 where the
executemethods forDbtLocalBaseOperator,DbtDockerBaseOperator, andDbtKubernetesBaseOperatorwere different.This PR refactors the
executemethod to theAbstractDbtBaseOperatorso it's the same for all of the local, docker and kubernetes inherited operators, and addsbuild_and_run_cmdas an abstract method since the implementation is different across the 3 different execution modes.Related Issue(s)
closes #804
Breaking Change?
None
Checklist