Make array_node_map_task the default map_task #2242
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
… map task module to `legacy_map_task` Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
eapolinario
requested review from
wild-endeavor,
kumare3,
pingsutw and
cosmicBboy
as code owners
dosubot
bot
added
the
size:M
Signed-off-by: Eduardo Apolinario <[email protected]>
eapolinario
changed the title
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2242 +/- ##
==========================================
+ Coverage 84.74% 85.98% +1.24%
==========================================
Files 315 320 +5
Lines 24142 24250 +108
Branches 3666 3667 +1
==========================================
+ Hits 20458 20852 +394
+ Misses 3025 2746 -279
+ Partials 659 652 -7 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Eduardo Apolinario <[email protected]>
dosubot
bot
added
size:L
Signed-off-by: Eduardo Apolinario <[email protected]>
…re loaded Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
eapolinario
commented
Mar 6, 2024
| @@ -19,6 +19,7 @@ google-cloud-bigquery-storage | |||
| IPython | |||
| keyrings.alt | |||
| setuptools_scm | |||
| pytest-icdiff | |||
There was a problem hiding this comment.
This gives better diffs in case of failures: https://github.com/hjwp/pytest-icdiff
Signed-off-by: Eduardo Apolinario <[email protected]>
wild-endeavor
approved these changes
Mar 6, 2024
3 tasks
austin362667
pushed a commit
to austin362667/flytekit
that referenced
this pull request
Mar 16, 2024
* Swap arraynode map_task Signed-off-by: Eduardo Apolinario <[email protected]> * Fix unit tests Signed-off-by: Eduardo Apolinario <[email protected]> * Add pytest-icdiff to dev-requirements.in Signed-off-by: Eduardo Apolinario <[email protected]> * Fix test_node_creation.py tests Signed-off-by: Eduardo Apolinario <[email protected]> * Remove array_node_map_task from experimental module and rename legacy map task module to `legacy_map_task` Signed-off-by: Eduardo Apolinario <[email protected]> * Lint Signed-off-by: Eduardo Apolinario <[email protected]> * Fix one more mention to legacy map task code Signed-off-by: Eduardo Apolinario <[email protected]> * Remove `--experimental` and include `--legacy` Signed-off-by: Eduardo Apolinario <[email protected]> * Fix flytekit-k8s-pod/tests/test_pod.py test Signed-off-by: Eduardo Apolinario <[email protected]> * Remove the `--legacy` flag and rearrange how the map task resolvers are loaded Signed-off-by: Eduardo Apolinario <[email protected]> * Remove map task resolver imports from entrypoint Signed-off-by: Eduardo Apolinario <[email protected]> * Fix name in k8s-pod plugin Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
2 tasks
fiedlerNr9
pushed a commit
that referenced
this pull request
Jul 25, 2024
* Swap arraynode map_task Signed-off-by: Eduardo Apolinario <[email protected]> * Fix unit tests Signed-off-by: Eduardo Apolinario <[email protected]> * Add pytest-icdiff to dev-requirements.in Signed-off-by: Eduardo Apolinario <[email protected]> * Fix test_node_creation.py tests Signed-off-by: Eduardo Apolinario <[email protected]> * Remove array_node_map_task from experimental module and rename legacy map task module to `legacy_map_task` Signed-off-by: Eduardo Apolinario <[email protected]> * Lint Signed-off-by: Eduardo Apolinario <[email protected]> * Fix one more mention to legacy map task code Signed-off-by: Eduardo Apolinario <[email protected]> * Remove `--experimental` and include `--legacy` Signed-off-by: Eduardo Apolinario <[email protected]> * Fix flytekit-k8s-pod/tests/test_pod.py test Signed-off-by: Eduardo Apolinario <[email protected]> * Remove the `--legacy` flag and rearrange how the map task resolvers are loaded Signed-off-by: Eduardo Apolinario <[email protected]> * Remove map task resolver imports from entrypoint Signed-off-by: Eduardo Apolinario <[email protected]> * Fix name in k8s-pod plugin Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Jan Fiedler <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tracking issue
NA
Why are the changes needed?
Array node map tasks have been present in the SDK for a long time. Time to rip off the bandaid and make them the default map_task.
What changes were proposed in this pull request?
A few changes:
map_taskto point toarray_node_map_task.array_node_map_taskfrom theexperimentalmodule.legacy_map_taskmap_taskresolvers are loaded, instead of relying on flags to instantiate specific resolvers, we use the resolver name to produce a valid reference.How was this patch tested?
Unit tests and local runs using sandbox.
Setup process
Screenshots
Check all the applicable boxes
Related PRs
Docs link